- Data on demand
- JSS and HTML customization
Owner at a tech consulting company with 51-200 employees
TIBCO Spotfire and the In-Database/In-Memory Analytics Choice
(First posted October 4, 2012 in the Breakthrough Analysis blog. )
Visual analytics leader TIBCO, with its September 25 launch of Spotfire 5.0 and announcement of a new Teradata alliance, wants analytics customers to have it both ways, promoting both in-memory analytics and, for the largest or deepest problems, push-down of calculations into the Teradata engine via a technique called in-database processing. Flexibility is good. In-memory processing speeds interactive-analysis response times which in-database analytics reduces data-access delays and calculation time, taking advantage of parallel processing by the the database management system (DBMS). For each of a large variety of analytical processing challenges, which approach works best?
On the one hand, the new analytics partnership — “Spotfire harnesses Teradata for executing complex calculations and predictive analytics in-database” – delivers “extreme data discovery and analytics.” On the other, newly launched Spotfire 5.0 is a “re-architected in-memory engine specifically built to enable users from across the enterprise to visualize and interact with massive amounts of data” that doubles down on the long-standing positioning of TIBCO Spotfire as “in-memory analytics software for next generation business intelligence.”
Next Generation BI and Database Systems
The next generation of business intelligence is indeed interactive and visual, typically involving iterative, exploratory analyses. Tableau and QlikTech, notably, compete with TIBCO on this front. And business is increasingly demanding real-time analysis of high-velocity data, without the latency involved in writing data streams to a database. These scenarios are tailor-made for in-memory processing. But here’s the rub. Next-gen BI also calls on huge volumes of diverse data and, often, the application of sophisticated computational algorithms, necessary to make sense of time series, geolocation, connection-network, and textual data. If you’re crunching high-volume data from social/mobile sources, and from certain species of machine/sensor-generated data, you may working beyond the responsiveness bounds of in-memory analytics… or you may not.
QlikView, for instance, can directly import social-sourced data via connectors from QlikTech partner company QVSource, and all three of the companies I’m mentioned work with ‘unified information access’ vendor Attivio to ingest data from a variety of unstructured sources that Attivio handles. And Teradata’s partner ecosystem includes text-analytics providers Attensity and Clarabridge, who software runs outside Teradata systems rather than in-database, just as SAP, in a tighter coupling, provides text capabilities via a data-services framework.
It’s a complex analytical-software world out there! We see that in-memory and in-database analytics occupy overlapping territory. How to choose the right approach in a bi-BI world? In what circumstances should you pull data from the external DBMS for those fast in-memory analyses — TIBCO touts the “two-second advantage” — and when should you push-down complex calculations and predictive analytics into an external database system?
TIBCO-partner Teradata provides a very worthy analytical DBMS, with parallel query processing, high reliability, low-latency data availability, broad data-management capabilities, and rich in-database analytics. So happens I wrote a paper for the company a couple of years back, Frequently Asked Questions about In-Database Analytics. (Teradata has paid me to write other papers, QlikTech is a consulting client, Attensity is a sponsor of my up-coming Sentiment Analysis Symposium – If you’re into social-media analytics, market research, or customer experience, check it out, October 30 in San Francisco – and I have done paid work for SAP and Sybase in the last year.) Teradata is not the only player in the game. Oracle and Microsoft SQL Server Analysis Services are additional external in-database analytics options with Spotfire, and DBMS options including EMC Greenplum, IBM Netezza, and SAP’s Sybase IQ all support in-database analytics unallied with TIBCO.
In-Memory vs. In-Database Guidance
A TIBCO-provided customer testimonial hints at one decision criterion. TIBCO references MGM Resorts International, a Spotfire and Teradata customer. “‘Being able to work with Spotfire directly connected to billions of data records through Teradata will greatly improve our ability to manage the Big Data dilemma,’ said Becky Wanta, Senior Vice President and Global Chief Technology & Innovation Officer,” as quoted by TIBCO. Focus on “billions of data records” and the word “manage.” In-database analytics in a centralized data store provides for shared-but-controlled access and robust administration, not only for data but also for database-embedded analytical routines, whether instantiated in SQL, custom code, or code libraries.
Consider another customer testimonial: Alan Falkingham, director, business intelligence at Procter & Gamble says, again quoted in the TIBCO release, “We are excited by the prospect of Spotfire 5.0 being able to efficiently analyze and visualize extreme data volumes by executing analytics directly within our database architecture.” Key-in here on “visualize” — visual analysis happens in the user-facing front-end, often as part of an iterative, exploratory process, where in-memory excels — where the efficiency is in handling heavy computations, that touch those “extreme data volumes,” close to the data, in the DBMS.
TIBCO Spotfire Vice President of Marketing Mark Lorion explained, “Being in-memory, there will still be limits based on specific machines configured and deployed… Our approach enables organizations to bridge between/distribute across the two architectural approaches to best fit the use case. This allows companies to leverage the benefits of in-memory freedom with the existing investments in data managed elsewhere.”
Still, how to decide what’s done in-database and what’s done in-memory? Lorien didn’t respond to a question I posed, asking the limits of the in-memory approach. I asked Gartner analyst Merv Adrian his take. It was, simply, the following: “I haven’t thought about it much. Anything that fits in memory ought to be done there, in my opinion. But the DBMS in-memory doing processing in-database would be ideal.” Ideal indeed, not currently doable so far as I know. SAP HANA is the most prominent in-memory database system, but while the HANA database has a column store option and handles multidimensional (OLAP) analyses, but I don’t believe you can embed serious analytics in-database. Similarly, Kognitio‘s in-memory analytical platform doesn’t support in-database analytics.
Putting aside capacity questions, analytical-routine availability is a key factor. You may not have a choice, if implementations of the algorithms you need are available or can be programmed in-memory or in-database but not both.
Heterogeneous Environments
Looking ahead, two premises I will state are that as tempting as it is to try to handle all work in-memory, disk-reliant analytical database systems — relational like Teradata and others I’ve cited, or NoSQL — will remain a corporate reality for a while to come, and that neither DBMS-embedded analytics nor in-memory analytics will completely meet enterprise BI needs anytime soon. Software companies such as TIBCO and Teradata, with duplicative predictive-analytics capabilities, would not partner if they were able to go it alone.
So you’re left with what we used to call systems analysis, and with experimenting, to see what works best given your own mix of data (volume, type, and pace), analytical routines, user workload, and management and administration requirements.
Enterprise IT environments are heterogeneous, meeting diverse demands. Flexibility and systems interoperability are musts for partial-solution vendors including both in-memory and DBMS-centered analytics providers. Flexibility and systems interoperability are must-haves for the earlier-generation analytics, the next, and the analytics generations after next as far as the eye can see.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
IT Analyst at a tech services company with 10,001+ employees
It has allowed us to host client and web on the same server.
What is most valuable?
How has it helped my organization?
It has allowed us to be able to host client and web on same server.
For how long have I used the solution?
I have used it for three years.
What was my experience with deployment of the solution?
The server crashes at times with plugins.
How is customer service and technical support?
Technical support is 8/10.
How was the initial setup?
Initial setup was simple.
What other advice do I have?
Use it for all of your analytical reports.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
TIBCO Spotfire
April 2025
Learn what your peers think about TIBCO Spotfire. Get advice and tips from experienced pros sharing their opinions. Updated: April 2025.
851,604 professionals have used our research since 2012.
Vice President - Fixed Income at a financial services firm with 501-1,000 employees
Visualizations help us dissect data more and spot outliers more easily. Their mobile solution and export functions are relatively weak.
What is most valuable?
Visualization is the most valuable feature. Visualizations help us dissect data more and spot outliers more easily.
How has it helped my organization?
We have a dashboard set up with Spotfire to show risk metrics across the firm for senior management.
What needs improvement?
Their mobile solution (i.e. to use on mobile devices) and export functions are relatively weak. Spotfire templates are typically designed and mocked up on PC and then got rolled out to users, who in most cases require the capabilities to view them on both PCs and mobile devices. Due to real estate issues, the PC versions wont’ fit well on the latter. Spotfire doesn’t offer easy solutions for the transition yet. Separate templates tailored to smaller screens have to be set up manually by users.
For how long have I used the solution?
I have been using Tibco Spotfire for about four years.
What do I think about the stability of the solution?
We have not had any stability issues.
What do I think about the scalability of the solution?
We have not had any scalability issues.
How is customer service and technical support?
Technical support is medium. They don’t have a user hotline to call. Any technical issue will have to be either directed to consultants or logged in a ticket on the web.
How was the initial setup?
Setup was straightforward. Users can use the product straight out of box.
What's my experience with pricing, setup cost, and licensing?
I was not involved in pricing and licensing.
Which other solutions did I evaluate?
We evaluated MicroStrategy.
What other advice do I have?
It is a great tool. Compared to other products on the market, Spotfire is designed for enterprise-level solutions. The key, however, to fully utilize the tool is to have an open/transparent data infrastructure to feed into the tool.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Integration Analyst at a tech services company with 1,001-5,000 employees
The predictive data analytics in 7.0 make it easy to build a report on the fly.
Valuable Features
- Information links, though a little complex, really make connecting to data sources reliable.
- The predictive data analytics in Spotfire 7.0 make it super easy to build a report on the fly.
Improvements to My Organization
Spotfire is great for data visualization.
Room for Improvement
Backend administration of the application can be difficult. There are constant upgrades, but it would be better to do just releases for smaller changes.
Use of Solution
I have been using it for one year.
Deployment Issues
I have not encountered any deployment, stability or scalability issues.
Customer Service and Technical Support
Technical support is average.
Initial Setup
We have a very easy and straightforward way of installing Spotfire at our company. Since it is a self-service tool, it is available for everyone via our IT store.
Other Advice
Spotfire is an easy-to-use yet powerful tool. Though the administration can get tricky and a little difficult, it is worth it for the efficiency and optimization it provides for our users. Overall, it is a stable and powerful application.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
TIBCO Specialist at a tech services company with 10,001+ employees
It can work with data using in memory mode, this makes the loading process faster.
What is most valuable?
TIBCO Spotfire can connect to many different data sources in order to acquire data, from many vendors such as SAP, Oracle, IBM, or even just local files.
It can work with data using in memory mode, this makes the loading process faster.
It is easy to customize, and extendable with custom functions.
A very useful API is provided in order to integrate it easily with any web application.
What needs improvement?
The developing and runtime environment are very stable. The TIBCO Spotfire architecture environment is not complex. However, not all the runtime components are multiplatform. To be exact, the component whose purpose is to render the data in a web browser can be installed only on Microsoft machines. This is a very big limitation from this point of view.
For how long have I used the solution?
The first approach was in 2010. I have used TIBCO Spotfire in four or five different projects as of 2016.
What do I think about the stability of the solution?
The TIBCO Spotfire product is really stable. No relevant issues have been encountered.
Most of the time the TIBCO Spotfire dashboard is used by business users. It was very common for them to notify us regarding weird behavior in business data patterns that was always ending with a discovery of bugs on the backend applications.
How are customer service and technical support?
Technical support was always very good and fast. Always leading to a resolution of the problem immediately.
Which solution did I use previously and why did I switch?
Together with TIBCO, my previous company was a partner of SAS. Recently, SAS provided a BAM and BI solution, but personally I have found it not comparable to Spotfire: huge memory consumption, no API available, not customizable.
How was the initial setup?
The initial setup is quite easy, a few steps to perform before the application is ready.
A little bit more complex is the configuration of SSO, but the documentation is covering all the scenarios in a very clear way.
What about the implementation team?
Implementation with TIBCO Spotfire consists basically of understanding the business requirements, and transforming them into readable dashboards and diagrams. Most of the time, in my case, implementation was made by a one-man team.
What other advice do I have?
TIBCO Spotfire is a new challenge, very different from any typical implementation a developer used to do. Most of the time, it consists of understanding the data. Very often, that requires direct contact with the business department of the customer.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Growth & Data Guru at a non-profit with 51-200 employees
It links to the database and the dashboard updates automatically. You don’t have to export a CSV file. Implementation could be much easier.
Valuable Features:
Real time analytics: TIBCO Spotfire links to the database and the dashboard updates automatically, which is great. You don’t have to export a CSV file to get a dashboard.
Improvements to My Organization:
The client where I have implemented TIBCO Spotfire was using it to understand calls received on their call centre. With the analytics provided, they were able to understand when during the week / day they receive most of the calls to then take action and get the right number of agents.
Room for Improvement:
I would say that the implementation of the product could be much easier that it is at the moment. It is a complex process at the moment to set up TIBCO Spotfire with your database; not always straightforward.
Use of Solution:
I have been using it for six months.
Initial Setup:
The initial setup is supposed to be straightforward, but I found that there is often some glitch in the setup that requires thought.
Implementation Team:
TIBCO Spotfire provides an expert to set up the solution for their clients. This usually takes 2-3 days.
ROI:
It is an expensive product, but if you have the right data analyst to use it, the ROI can be quite high. This number depends on a lot of factors. How much analytics are you doing at the moment? How good is your data analyst?
Other Solutions Considered:
I have used Tableau and Alteryx, which are similar solutions. I would say that TIBCO Spotfire is a combination of those two other solutions, although I think Tableau is much more user friendly.
Other Advice:
I think you should get a trial and see what are the possibilities and understand who would be using the product. Define your business goals first and then use TIBCO Spotfire in order to measure those.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Hola,
Gasping at your brilliance! Thanks a tonne for sharing all that content. Can’t stop reading. Honestly!
i am testing ADB adapter 7.2 for 40 Million data processing.
As adb 7.2 supports Multithreading in Publication Service we want to use this feature.
but during testing we found that there is "JVM thread count" as well configurable during deployment of ADB adapter.
we are configuring "Number of Publication Service Thread" as 100 but keeping "JVM thread count" default as 8. In this case which will take precedence, will the Number of Publication Service Thread override the value of "JVM thread count" or we have to keep both these threadcount value same. we want 100 threads used by adapter(Heap is 4 GB).
It would be great if we get some clarification quickly.
Great effort, I wish I saw it earlier. Would have saved my day :)
MuchasGracias,
Mark Hetch
CEO with 51-200 employees
Spotfire, Tableau and QlikView – in a Nutshell
Verdict
Pick Spotfire if your analysis is likely to become complex as time progresses. Pick Tableau if you primarily want to satisfy the less complex needs of business users, and choose QlickView if you want a broad architecture that satisfies general needs.
Spotfire from Tibco provides an easy to use interface for data visualisation, analytics and the creation of dashboards. Most of the slicing and dicing can be done through drag and drop and a multitude of intelligent functions (eg scaling the time axis on charts automatically) make light work of many analysis tasks. The lightening fast execution speeds are also a great advantage, particularly on large data sets.
More complex analytics can be accomplished through the R programming language, and the R runtime engine has been embedded into the Spotfire statistical server. This allows R based analysis to be fed out to as many users as required (typically through its WebPlayer web client).
Version 5.0 of Spotfire has embraced big data and particularly in-database analysis, with support for Oracle, SQL Server and Teradata – others will follow.
In a nutshell: Full blown analytics and visualization environment with both end-user and analyst functionality.
There are three elements to the Tableau product set. Tableau Desktop provides a drag and drop analysis environment that is highly tuned for productivity. The product is capable of handling very large amounts of data and supports extensive collaboration features. Dashboards can be created from multiple analysis and these can be shared if desired. The technology is designed to connect straight to data sources without the usual extraction phase.
Tableau Server provides browser based analytics and the construction of dashboards which can be filtered and drilled in to by other users with relevant privileges. Mobile devices are also supported including iPad and Android devices. Finally Tableau Public is a web based service supporting the publishing of interactive real-time graphics on web sites. Embedding the graphics is as straightforward as embedding YouTube videos.
In a nutshell: Excellent environment for business users with a minimum of technical fuss to get results.
QlikView majors on in-memory processing for high levels of performance, and substantial inbuilt intelligence to maintain associations between data, compress data (by as much as 90%) and aggregate data on the fly. The product architecture has been designed to provide functionality for IT, analysts and end-users.
Business users can use web and mobile clients through the QlikView portal. Analysts perform analysis through QlikView Desktop and IT professionals use the QlikView Management Console. The QlikView Server (QVS) is the hub of the QlikView archiecture and supports all three users though its very fast in-memory processing.
In a nutshell: Innovative and highly productive environment for end users, business analysts and IT professionals. Extensive inbuilt intelligence provides a very productive environment.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Hi Chidambaram,
The Spotfire Analyst and Spotfire Desktop clients have an embedded R engine (TERR). There is no need to separately install TERR. The main purpose of using server-side TERR is the ability to run R using a larger computer on larger data sets.
Best,
Steve
Consultant at a consultancy with 1,001-5,000 employees
I have used it to develop visualizations that have identified historical trends that allowed an organization focus its operations to reduce injuries.
What is most valuable?
- Ability to handle large amounts of data from multiple sources and quickly visualize it for users.
- Connectivity to widely used data management (MDM) solutions in place in most organizations
- Easy to train users in how to use the dashboards, visualizations and maximize their performance as users
How has it helped my organization?
From a safety standpoint, I have developed visualizations that have highlighted trends in past history. When identified, they helped the business determine where to put more focus and consciousness in operations so that they can reduce injuries.
What needs improvement?
I would like to see improved mobile capabilities.
I would like more functionality in the formatting of the pages so that they can have more of an application with the look and feel of other custom-developed solutions with the organization.
For how long have I used the solution?
I have used it for eight years.
What was my experience with deployment of the solution?
I have never had issues with deployment or scalability, TIBCO offers different versions to fit customers’ needs (desktop, web based, etc.).
How are customer service and technical support?
Technical support is average; however, the user conferences that are held regionally and nationally provide a lot of value.
Which solution did I use previously and why did I switch?
I previously used Oracle OBIEE. We switched because:
- Development time was too long.
- Overhead of resources required to maintain it was too high.
- Spotfire will offer very fast enterprise data and visualization capabilities in a short amount of time.
How was the initial setup?
Initial setup is straightforward; there are downloads that you perform in order that make it easier than it used to be, but it is not quite plug and play for an enterprise implementation.
What about the implementation team?
I implemented it with an in-house team.
You may not need one, but establish a contact number at TIBCO for extra help should you encounter problems.
What was our ROI?
ROI depends on the maturity and adoption of the organization. Spotfire can have an exponentially higher ROI than what it costs, if implemented well, adoption programs are in place and users are trained.
What's my experience with pricing, setup cost, and licensing?
Users can license and download directly from the TIBCO website and be active within minutes.
What other advice do I have?
I highly recommend Spotfire.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
Download our free TIBCO Spotfire Report and get advice and tips from experienced pros
sharing their opinions.
Updated: April 2025
Product Categories
BI (Business Intelligence) ToolsPopular Comparisons
Microsoft Power BI
Tableau
IBM Cognos
Amazon QuickSight
SAP Analytics Cloud
SAP BusinessObjects Business Intelligence Platform
Oracle OBIEE
MicroStrategy
ThoughtSpot
Buyer's Guide
Download our free TIBCO Spotfire Report and get advice and tips from experienced pros
sharing their opinions.
Quick Links
Learn More: Questions:
- What's your experience or opinion about Spotfire vs. Tableau vs. Qlik?
- Tableau vs. Spotfire - What do I need to know regarding pricing and usability?
- Spotfire vs. QlikView. What can one do which the other cannot?
- A journalist is writing a story about which Data Visualization software product to choose. Can you help him?
- How would you decide between Microsoft Power BI and TIBCO Spotfire?
- What challenges to expect when migrating multiple dashboards from TIBCO Spotfire to Microsoft Power BI?
- Which is the better solution and why - Oracle OBIEE or TIBCO Spotfire?
- How would you compare TIBCO Spotfire vs Microsoft BI?
- How to integrate Jira and Aha! to TIBCO Spotfire?
- When evaluating Business Intelligence Tools, what aspect do you think is the most important to look for?
I used Spotfire few months ago and I noticed the following pros: it is a very intuitive design tool, once the user learn one thing, it's quite easy to use that knowledge to do something else. Marking visualization components is very intuitive and easy to understand. The tool is also very easy to customize. On the other hand, the GUI is poorly designed and difficult to use.