WhereScape RED Reviews

We are a typical manufacturing company. We have around 65 to 70 manufacturing locations all over the world, and we hold down 60 different sources of data coming from ERP, CRM, and other applications kind of thing. We have eight different data source types, such as Oracle, SQL, et cetera. It really helps in terms of automation. We have a template and we have data type matching in place. It's even configured to be a drag and drop and helps you to segregate all the objects in types of projects and tools.

We were a division base for the department under the warehousing team, and have been moving to the centralizing and standardizing of some aspects, and combining all the individuals doing warehouses into one centralized enterprise-level warehousing team. The solution allows us to streamline users so that fewer people are needed to do the warehousing for the entire company.

It's easy to deploy. Everything is metadata-based and we can do work by location also. It's easy to segregate between teams.

WhereScape is really helpful in terms of architecture data. Everything is one of automation. Two people can do thousands of tables in one day or two. It saves a lot of time.

The solution is very stable. 

It's compatible even with old methodologies.

The solution is able to automatically generate code for you for managing procedures. 

The product is very flexible. 

You are able to easily create statements.

The initial setup is quick and easy.

The scheduling part I don't like due to the fact that it allows you to schedule as a parent and child and other things, however, the error trackability has to be a little more user-friendly. 

It's also not user-friendly in the sense that it loads all the jobs and there are not enough filters so that it doesn't need to load everything.

If the job fails, you don't get any type of alert or email. It would be ideal if there was some sort of automated alert message. 

Technical support isn't the best.

It would be ideal if we understood how to do it in a card exception regarding exclusion, where the card is captured separately rather than filling the whole process on the data inbound side. Certain workloads like this are organized in such a way where you seem to be doubling the work as opposed to streamlining the process. 

I've been using the solution for two and a half years at this point. 

The stability capabilities are really awesome. It doesn't crash out. It also doesn't do sluggishness or anything. It's fast and stable.

The solution is scalable in that you can have multiple people working on it at once and it won't slow down.

The data scalability depends on the methodology you use.

Technical support isn't the best. They seem to be more money-oriented than customer-oriented. We found that after the purchase, assistance and support really dropped off. 

Previously, we were completely on the Microsoft stack. This product gave us a bit more automation.

The product is easy to deploy. It's not overly complex or difficult. 

It only takes one person two or three hours to complete the task. It's very fast and simple.

The deployment was handled in-house. We have many experts on staff. We did not need any outside assistance. 

We tend to use the latest version of the solution.

We are considering the cloud. We are currently debating whether we need to go to WhereScape or drop WhereScape and we go to the cloud. I've been asking about their host-ability and version controllability if we move to the cloud and they were not giving too many promising answers. Basically, when we go to cloud viewer, we want it to go completely on a platform as a service, not infrastructure as a service, as IaaS is a little more costly than PaaS - The Chinese PaaS. 

Also, we would like to use, as a team, seven or eight people at scale - not just on the server installed and then use it as one or two accounts. The whole stability issue on the cloud needs a little more clarification.

I'd advise companies considering the solution that, if it is on-premise and they have a good volume of data coming, it can be easy to start using it. I'd say that negotiating is important and making sure that the pricing is clear, as they can come back and alter the price after the fact. 

Other than that, if you had a lean team, I would say go for Wherescape. It's simple, straightforward, and not too complex. That said, if you have more XML semi-structured data, or unstructured data, you might want to look for some other product.

I'd rate the solution at an eight out of ten.

My primary use case is Data Warehouse & Mart Development plus data integration and content generation applications. We operate in many environments including Teradata, Oracle, & SQL Server.

Using this solution sped up development and allowed for easy maintenance.

The most valuable feature is the metadata generated code and data structures and that it is a fully integrated environment. This has many benefits. It allows for rapid generation of ELT without writing code thus less errors, tracking consequences of changes, tracking transforms, improved ease of maintenance, it uses the data structures directly, it is drag and drop with dialogue entered information. It also automatically includes full audit capabilities.

Project-based searching of data objects in the data warehouse browser needs to be improved. When processing stage (work) tables if there are a lot then one has to scroll through a larger list. There is an ability to have projects or subject areas for processing that if applied to source tables would speed development.  

On a separate note - those who have criticized this project for having a row based cursor and thus inefficient approach have not understood how the product works. I have used it for 10ish years and >99% of my processing is set based and efficient and supported by the product. It explicitly asks for set based vs row-based processing.

I have been using WhereScape RED for thirteen years.

Very - it is mature and has remarkably few defects.

Used it on a Telco enterprise data warehouse using Teradata. As it uses database processing (ELT) it is dependent on the underlying database. It can scale to very large volumes with the appropriate underlying database.

Good.

I used SSIS - it struggles once the volumes increase substantially, has relatively poor error handling and misses many standard transforms.  I was BI qualified with Microsoft.

Also used BO data services - not an expert but found job management to be exceedingly poor.

Simple. Install - set up ODBC connections, database links to source and install which automatically installs metadata database and job management scheduler. 

Multiple implementations and a mix of teams. Generally vendor teams more successful as they understood the product principles and applied these rather than worked in their former fashion.

Not sure anymore around licensing costs as the product has been purchased by Idera

SSIS, BO Data Services, Oracle stack, Cognos

For the cost it has a very effective bang for buck. See comment about new owners. 

I used WhereScape RED to do a proof of concept and understand how it worked.

I like the data vault implementations. It's useful. I also like the ELT feature. I also like the automatic generation of logic as it saves time. I like anything from the beta world, and that's why I took a look at this product.

Customization could be better. When you need to modify or write some custom code, it can get a little messy. Performance can also be better. It would be nice to have synchronicity when integrating multiple tools.

I've been using WhereScape RED for a year or two.

I received very good support from the Asia Pacific team.

We have used IBM InfoSphere DataStage and SSIS before.

The initial setup is fairly straightforward. It was a little challenging because we needed additional access.

Our company purchased a corporate unlimited license.

I would recommend this product to new users. 

On a scale from one to ten, I would give WhereScape RED a seven.

WhereScape Red helps to speed things up. It has many useful features and version 8 is  reliable. When I first started using it after having worked with Cognos Data Manager, I didn't really find it to be an enterprise-level tool because it was still very flaky. In fact, it had quite a number of bugs. It would fail on every second thing. If you are using it for some basic tasks, it will work fine, but as soon as you try some of the more complex features it would fail. I've seen a lot of improvement in the new versions.

Among the features I found most valuable is its support for Kimball Methodology and data warehousing best practices.

The schedular needs improvement 

I don't think the solution is very stable. I find that once you use this tool, you would like to have it in your tool kit. Even though I like this tool numerous bugs made it difficult and unreliable . Several of them have been fixed in the new releases. At an enterprise-level, you need a lot more. 

The scalability seems okay. It handles most of the data for our 30 sources. We are giving each source an independent repository and a separate EDW repo that uses synonyms for the target objects in the source repos.

We currently have six developers or data engineers using WhereScape, but we don't plan to increase the number, because training is not very cheap.

We have the advantage of having very close access to WhereScape's technical support. It was easy to get them to come on-site and answer many of the issues that we've had. That was quite a significant advantage.

I found the initial setup very easy. In fact, after one month of using it, I was able to have an independent setup on my own workstation. that shows how easy it is to set it up.

I started using this solution with MPI (Ministry for Primary Industries) .They have been a long-term customer of WhereScape Red. When I joined them, they had about 5 WhereScape Red users.

I would not hesitate to say that if someone has really limited time for a project it can make a difference and you can have the data ready for use in a very short period of time. This can be a lifesaver. I will rate this solution a seven out of ten. 

ETL processing in SQL Server into structures which are suitable for supporting business reporting needs. Automation of the importing and processing work.

WhereScape RED has improved our business's ability to generate needed reporting without requiring a large team of developers to manually code all of the necessary plumbing. WhereScape does a good job of handling this on its own.

WhereScape's deployment package is a fantastic feature. The application allows for selecting specific objects that you would like to deploy from one environment to another rather than deploying the entire database. This allows us to easily work as an agile team. At deployment time, WhereScape determines how to make the objects in the selected destination match the objects in the deployment application (through create and drop, then create and alter, etc.). Therefore, regardless of the state of the destination, we can be confident that the deployed objects will match the objects that we intended to deploy without disturbing anything else in the database. This is a huge benefit to our process because we can quickly create these packages, handing them off to teams who will be deployed to testing and production environments. All they have to do is deploy the package as-is.

The scheduled jobs which are run by the WhereScape scheduler seem to be a strangely separate animal. Unlike all other WhereScape objects, jobs cannot be added to WhereScape projects. Also, unlike all other objects, jobs also cannot be deleted using a WhereScape deployment application.

Our primary use case is to extract data from multiple heterogeneous data sources, stage and transform data as the business requires, and load the results into our enterprise data warehouse.

RED has provided us the ability to integrate, stage, and transform data from diverse sources into an enterprise-grade data warehouse which meets the needs of my organization, but it also enables us to easily and quickly make ETL or DW changes. It also integrates new data sources that the business requires.  

While there is a learning curve, (just as there is with any software), once you get the hang of RED, it is a breeze. It enables us to focus on the business requirements at hand than the tool that we are using to solve the problem.

Documentation (both user and technical): With so many moving parts, it is hard to keep track of the gazillion databases, tables, columns, and sources. RED generates comprehensive documentation and regenerates it as quickly as things changes, but it also provides impact documentation (quickly indicating where and what objects derive from and where they are consumed downstream).

Jobs cannot be deleted via the deployment package. When deploying from dev to QA or production, a job has to be retired. The job has to be manually removed from the target environment.

As far as service, I have sought their services a handful of times, and like the old adage (do not fix it if it is not broke), I can't think of any improvements here.

Their support staff are very knowledgeable, courteous, and professional. I feel their support staff go above and beyond to assure their customers are satisfied. 

Quickly develops a data warehouse for our organization with documentation and can track back/forward features.

Speed to market of a warehouse solution at a relatively inexpensive price point.  We used a consultant to get us started, but the tutorials offered by product were very helpful as well.

Documentation and the ability to track back/forward from fields, processes, etc. Data transformations and rollups are easy to accomplish.

A more robust support center. It has been a bit difficult to find solutions to problems that are out-of-the-box. It would be nice to have user groups available on their website. 

The ability to execute SSIS projects within WhereScape would be nice because we have a lot of packages that are too cumbersome to recreate.

Producing data warehousing solutions quickly with this DW automation tool. You can generate DWs in many types of formats. We prefer Kimball architected data warehouses.

It has allowed us to produce consistently created, well-documented, easy to maintain DW solutions for our clients.

This is a fantastically, robust DW tool that will make you at least 10 times faster in producing a DW.

• It does all the code generation for you. It has very easy code to read, and it is efficient.
• Auto documentation feature
• Plenty of additional custom documentation features
• It has a built-in automatic scheduling environment.
• Naturally produces a way to easily debug your DW data solutions.

Here are some things that could improve the overall experience:

• A tool to diagnose what is missing from the environment for WhereScape to install successfully.
• Improve the object renaming ability (it works, but it could be more automated).

Never.

No issues.

Support is absolutely excellent, efficient, and timely.

• DataStage
• SSIS

Setup is straightforward, but sometimes the environment being installed does not have some of the basic things the system needs for WhereScape to install properly. 

WhereScape might want to consider evaluating the system for missing things that need to be in place before the install and recommend how to fix those situations to make a successful install.

It was implemented in-house.

At least 10 times.

No.

Enterprise Data Warehouse and enterprise data staging.  Automation and documentation of all data warehousing processes.

It did not pass usability, reliability, sustainability or performance tests so there was no potential benefit other than minor documentation advantages over the current automated documentation solution.

Documentation has excellent potential as it provides the capability to generate data lineage and a complete set of rich documentation and output in multiple formats.

The tool supports multiple target update methods.

Support for multiple design methodologies including Data Vault.

In my opinion, the tool requires a much different architectural approach to be effective. It has fundamental architectural flaws that prevent it from being a practical solution for even moderate volumes of data.

The largest issue is that it violates the most basic database data-movement physics through its reliance upon the infamous RBAR or "row by agonizing row" cursor operations within Tsql. In contrast, efficient database apply operations to whole sets of data rather than row by row achieved through Sql joins and set based queries. Note that this is a common problem with code generators as they end up trading scalability for flexibility. Ironically this also makes them unable to take advantage of the much touted "push down" operations that is one of their big selling points. This tool avoids the traditional lookup ETL lookup cache component issue (sometimes referred to as blocking transactions) - that much is true.  However it ends up doing something similar but just within the database and with no easy way to override it.  Experienced ETL developers know not to do any blocking transactions or RBAR if they can possibly help it. And with traditional ETL tools you either take advantage of built-in push-down capabilities or pre-join the data in views or data sources to avoid this kind issue.

Other constraints include...

- Repository only supports a single target database

- No support for change data capture or delta detection - that must be custom coded

- Must be used to stage data or it requires a redundant copy of staged data.  This ended up more than double our storage

- Relies 100% upon target database server with no ability to scale ETL to separate server

- Relies 100% on the target database engine to do all the work (along with everything else the database must do like reporting)

- Cannot acquire data from sources other than flat files. It relies upon the use of linked servers (which many shops don't allow) or calling an ETL tool such as SSIS in the background for data movement. In that case you're still relying on an ETL tool for the ELT portion so then you're actually using two different tools - Wherescape plus a 3rd party ETL tool instead of just one tool.

Specific list of areas for improvement:

* Performance (this is the area that needs the most improvement)

* Efficient storage utilization

* Logging

* Count validations (data movements don't validate counts which is 101 basics to prevent data leaks)

* Scheduler reliability and transparency

* Ability to separate data acquisition/staging processing from target schema population

* Providing automatic record count validations

* Support for change data capture

* Support for in-line transformational procedure language that extends beyond database procedures

* Documentation lineage accuracy and completeness

* Support for sharing metadata repository across multiple targets

* Improvements in staging architecture requiring fewer copies of source data

* Supporting transparency into post-gens

* Providing encapsulated pre and post events for custom code that doesn't get wiped out when you re-generate supported code from the tool.

* Better support for task orchestration in a top-down way that can be easily understood.

* Support for direct parameter passing from job schedulers.

The scheduler often hangs with nothing to inform the user why it isn't working. Sometimes this is simply due to a developer starting a scheduler from their PC which integrates with the repository but isn't obvious when looking at the shared scheduler. Other issues seemed to be only resolved by re-generating the process from scratch and hope for a different result (which often there is), especially if the scheduler hangs. It is very easy to choose an incorrect option when generating processes only to result in something that just hangs with no error message. Then you're knee deep in database procedural language trying to debug a tool's generated code.

The sheer resources that it consumes easily overwhelms a server and causes the database to freeze due to exceeding resource constraints. The tool generates an unexpected and seemingly unpredictable amount of staging data (both in and out of the database depending on data movement option chosen) consuming very large amounts of disk space with redundant copies of data. This tool will not promote friendly relations with your system administrators.

The product was deemed unscalable in terms of data storage and performance.

Customer Service:

Very good

Technical Support:

Very good, although this was a POC so I can't speak for production support after the product is purchased.

Currently use a traditional ETL tool that meets our requirements. The test scenario involved around 250 GB of source data with a resulting 50 GB (compressed) star schema target. The ETL tool can talk to all our sources including multiple cloud sources using SOAP and REST web services and this tool is just a database procedure generator with no adapter capability. It instead relies on ETL tools to do that work so it doesn't eliminate ETL tools ending up having to support not one tool but two tools - both in licensing, support and expertise which was expected to cause significant increase in total support costs.

The tool relies on ODBC connections requiring complications in security in an AD environment over using OLEDB.

It requires a developer to either access the server directly or setup file shares. It was often confusing when doing development from a PC but working with a scheduler on a server and knowing what files need to be available from the PC verses from the server. The tool requires a great deal of attention to disk usage and location especially when using native database load utilities as it unloads all the data first into temporary files during the load which can easily consume your servers available space in unpredictable ways. Getting the environment functioning just for the training class was unexpectedly difficult.

The metadata repository is unique to a single target database and cannot be shared with any other target which prevents use beyond a silo datamart. This means data lineage is a silo repository for each target.

Software vendor was the lead implementor.

The ROI was expected to be very negative to the point of failure.

Factor in the price of specialized consulting who know this product. They're hard to find and expensive.

The existing traditional mainstream ETL tool and the previous one have been functioning without problems but the concept of a self-documenting tool that provides a common framework and shared metadata repository was a compelling alternative. This tool was already in use in our shop in a limited way so it begged the question on whether it could serve an expanded role.

Advice: Before choosing any tool it is best to do your homework and compare features to competitors and test the tool in a real life volume scenario. Software is never what its advertised from any vendor. Don't take other's word for it, especially if they have a vested interest in you using the tool. I know of testimonials about this tool and others that are based on wishful thinking and not reality. Caveat Emptor my friends. So the real question is how far off is it and does it have show stoppers that will prevent it's use in your environment? Fail to take this precaution at your and your organization's peril.

The compelling selling point for this and similar tools as "datamart in a box" providers is that they are pre-built frameworks and documentation. The opportunity is that staff with little data warehousing experience can get up to speed easily and not have to worry about building logging, orchestration and error frameworks or even know much about data warehouse design.

Unfortunately this tool's basic architecture has severe flaws which make it likely impractical for most real world marts or data warehouses. It has some significant cons in the following areas:

- Metadata support specific (cannot share across multiple) target database

- Unreliable lineage

- Lacks capability to interface with application API's

- Lacks out-of-box CDC control capabilities

- Poor/outdated GUI (looks like early 1990s)

- Lack of available resources in the market place who can operate this tool

- Lack of customer base and information available

- Primitive logging and error handling

- Forced duplication of staging data multiple times

- Inability to separate source data capture from targets (tightly couples sources with targets)

- Stores all objects in single target database causing number of issues such as backup/recovery, fragmentation, limited organization options for objects

Automated creation of documentation, quick time to market, ease of use, code standardization and support for Data Vault 2.0.

We can focus on the data itself (data quality, MDM, metrics) instead of spending time building code to move and transform the data. During the POC myself and one WS resource were able to deliver a project in 3 days that had been in development by other resources for 6 months and still hadn't been delivered.

We were more productive from the moment we started using RED but that X-factor has only improved as we have become more comfortable with the toolset and as RED adds new features with each successive release.

Since WS supports Data Vault 2.0, I would like to see automated support for hash keys on the appropriate Data Vault tables. WS does have a table trigger that can be manually added to the WS repository that will accomplishes this but it isn’t built in to the IDE. I would like to see multi-database (creating/storing tables in different databases based on type, e.g. staging, data mart, etc.). Similarly to the hash keys, it can be accomplished (I'm using multiple databases) it just requires making a manual entry in a RED repository table for each database. Again, having it officially built in to the IDE would be great. The only other thing I would note is that the IDE isn’t as intuitive as I would like. Like most tools, after several weeks of use it makes a lot more sense…but for new users there will be a moderate ramp up time.

About 1 year.

No.

The releases have been of surprisingly high quality compared to other vendor software I have used. We had the opportunity to test out a customer preview (i.e. beta) and were surprised to find no stability issues nor bugs in our testing. The features that were exposed in the customer preview were stable enough we felt comfortable moving forward with upgrading our Dev environments to begin development work using those features.

I have not run in to any issues with scalability at this point. Although I don't expect code-generation tools to write optimal code. I am surprised to see WS generate fairly good SQL code. I am very skilled in writing SQL and I was very impressed with what was generated.

Previously, I felt that I might need to write custom procedures to handle large table loads (i.e. 500+GB in a table), however, I no longer believe that to be true. I haven't reached those sizes in this environment (yet), however, I believe columnar indices and other standard SQL features would be able to cover any performance concerns since RED is producing well written SQL.

Good. Searching on the customer forums is very well done and has helped me not only solve issues I was running in to but also discover other features I wasn't aware of.

Good.

1. Information Builders iWay Data Migrator

2. SQL Server Integration Services

3. Hand coded SQL procedures

Data Migrator is just another ETL tool and the UI is not intuitive. Multi-threading is the only advantage Data Migrator has over SSIS and hand-coded procedures, however the cost isn't justified.

WhereScape RED handles the versioning, building of the objects (SSIS packages, SQL procedures, etc.) and provides an advanced, multi-threaded scheduler that makes it easy to manage object dependencies and objects that can be loaded in parallel.

A REALLY COOL feature is to create a track-back diagram from some destination object, e.g. FactSales. That opens a tab with a pretty diagram of the flow of data from the source system all the way through to the fact table. Tell RED to create a job from the diagram and *boom*, you have a schedule with all tasks and dependencies created to make sure that FactTable is loaded appropriately.

I felt the setup was fairly straightforward. I've seen better and I've seen worse. As with most things, it's just wrapping your head around the architecture and methodology WS has implemented.

In-house.

I believe the break-even on the investment is less than 1-year at this point.

Because WS builds native SQL code you don't have to worry about vendor lock-in as with other solutions. If you decide to walk away from WS you still have a working data warehouse environment. The only thing you would need to do is schedule the load procedures with another scheduling tool. Licensing negotiations are pretty straightforward as the development IDE is just a seat license model. They are very flexible in how they approach licensing.

I have used SSIS, Information Builders' Data Migrator and home grown SQL Server stored procedures. If I was to order these solutions in terms of ROI they would be:

1. WhereScape RED

2. SQL Server stored procedures

3. SSIS

4. Data Migrator

No solution is perfect and I could probably find nits here and there, however, I find WhereScape RED to be exactly what I was looking for. Roughly 80% of a data analyst's job is preparing data and 20% is doing the analysis. I purchased WhereScape RED to flip that equation on its head.