Pentaho’s Road To Profitability - Take 2

Tom Barber, a collaborator of mine, one of the most active community member of the Pentaho project, posted a very interesting blog entry this week. He is looking back at the path of Pentaho Corporation, a commercial open source business intelligence company, and their latest strategies for growth. He notes that in the past months, Pentaho has put a lot of effort in marketing initiatives and hired many big wigs in their marketing staff. In his opinion, this is somewhat against the ideals of commercial OSS development.

The commercial open source business model is still very young and has not encountered many great challenges up to now. Nor has it sailed in very troubled waters. Yet some signs are already warning people to be very careful with the years to come. Sun Microsystems (now Oracle) was surviving thanks to donations. Compiere recently made the news for all the bad reasons. Red Hat is doing pretty well though. In a nutshell, anything is still possible; good or bad.

Tom’s wish was that Pentaho would rather focus on paying skilled engineers rather than sales people. Now, as much as somewhat agree with the general idea, one must keep in mind that there is no correlation between the number of talented people getting paid to be on a project and it’s success. Some notoriously successful projects depend almost entirely on it’s community base. Mozilla Corporation to name only one. Others employ thousands of employees, yet achieve mediocre results. One could also argue that an effective marketing campaign will in fact boost people’s awareness, thus getting more talented people to join the community base. There is no tested and fail proof recipe so far. As I said earlier, everything goes.

I worked for the past months for a commercial open source company, and the same questions and uncertainties were part of every day discussions. How can a software company making no revenues on licensing be profitable? SQL Power Group, my employer, sponsors the only open source data modeling tool that works cross-platforms and offers, for free, the majority of the functionality included in widely known proprietary tools like ErWin. There are on a weekly basis between five hundred and a thousand downloads of SQL Power Architect. This is a lot for an OSS project in such a narrow niche. Yet a very small proportion of those are actually paying for support/consultancy, or making donations.

There are a lot of factors in play. First off, people willingly using OSS have overall better technical skills than others. To be fair, not every OSS is of acceptable quality, but you get to try as many as you want. Proprietary software is not better in any way, but you won’t get to try many of them, short of having very good contacts and/or deep pockets. This is one reason why reaching the tipping point of adoption is very hard for an OSS company. The percentage of OSS users ready to pay instead of figuring things by themselves is low. Way low. Waaayyy loooww. If you paid for software, you want support (and vice versa). You feel obliged to use the software because you already invested much in it. (Yes, money is time, time is money.) If you didn’t pay for the software and you hit a snag, you are far more likely to stop using it altogether. By abandoning it, you feel like you are exercising damage control, since you save time (thus money). You didn’t pay so why invest time? I’m fully aware that once you do the math, you realize that it’s the exact opposite. People usually don’t.

What does all that have to do with Tom? Well first thing first. Starting next week, I’ll be an employee of Pentaho Corporation. Yup. Am I a big wig? Nope. Am I a marketing guy? Well, I’m good looking, but not good enough with oxymoron. Am I a marketer? I’m having trouble making this blog post interesting, so nope, not a marketer either. I’m just some skilled dude who got his hands dirty for years and now decided that he would devote some serious time to make things better.

Look at it this way. In Tom’s vision of Pentaho, my ass is on the line. I deeply respect his insights on the market and software development in general. Yet here I am, proud as a peacock on a Sunday brunch, about to be part of this fantastic project. Am I worried? Hell no. I’m very enthusiastic about Pentaho having enough budget for marketing initiatives. This is exactly what OSS companies need; market awareness. People need to know you exist and that you are just as good as all those Fortune 100 companies. Even better than these. It has nothing to do with being a sell out. Marketing is not bad for your grassroots values, unless you make it so. Then again, you have only yourself to blame.

OSS companies need to reach the critical mass of contributors and adoption needed to survive. There is simply no other way. OSS users are picky, grumpy, bitchy and unforgiving. I know that for a fact. You want to thrive in that market share? You need two things. Proper marketing and skilled engineers. I’ll be doing my part in the later, and I’m very confident that the good people at Pentaho have picked skilled people to cover the former.

We are competing against giants, so let’s give them a ride for their money.

PAT 0.3 - Integration and nifty features…

Pentaho Analysis Tool sprint 2 is now over and released since a few weeks. We're currently in the process of wrapping up the third sprint. What's new? What new features will there be?

Nothing very spectacular really. Yet there are a few nice features worth mentioning.

• Integration with the Pentaho BI server
  PAT can now run embedded in the Pentaho User Console and be configured remotely. It only supports XMLA connections for this first draft, but don't worry; more compatible connection types are to be supported in the medium term. This also means that you can seamlessly use any XMLA provider. Sql Server? Essbase? Mondrian? You choose. All that thanks to olap4j. If you want specific details on this, Gretchen Moran has written a bunch of documentation on this. Congratulations to Paul Stoellberger and Gretchen Moran for this feature.


• Create multiple queries at once
  This was a requirement that was passed to us by Pentaho's engineering team. People want to build and use more than one query at a time without concurrency issues. That was not properly supported by the age old JPivot application. Now and then we encountered some problems with multiple queries, so this was something pretty high on our features list. The backend supported that since sprint 1 , but there was higher priorities for the GUI components.


• OlapTableModel first draft
  The Java API doesn't include a proper TableModel for OLAP data, so we're planning to write one. We still have lots of things to figure out on this subject, but we're planning to mock a draft specification before the sprint 3 deadline.

Our initial intent for the third sprint was to make the GUI nicer, but we're still trying to figure out if XUL would be a better generation tool than SWT. We are waiting for developments between the GWT-Mosaic team and Nick Baker of Pentaho and hope that all those nice gents would team up and cook something up to their talent. Fingers crossed here. :)

Also worth mentioning,  we started talks to review the Query model currently residing in olap4j. This is a soon to be major component as it will provide developers and GUI enthusiasts with a properly abstracted API to build queries against a data warehouse. Olap4j is  very great API for low level stuff, but it still needs an abstraction layer for the common folks. Building queries should not require in-depth knowledge of MDX, for you cannot expect all business analysts to master MDX anyways. Anyone who wants to participate in the process is warmly invited to manifest himself. We are really looking for input on this, whatever your background is. (As long as it's related to BI I guess...)

Pentaho Analysis Tool - Final push for sprint 2

I've been really busy lately working on sprint 2 of Pentaho Analysis Tool. We almost reached all our goals for this sprint and are hoping to wrap it all up in the next two weeks. There is still time to add a few late requirements in this sprint so if anyone has a very special wish, now is the time to express it.

About Pentaho Analysis Tool (PAT)


PAT is an attempt to replace the good ol' JPivot application, widely used in the Java world, as a web based browser for OLAP data. There are quite a few similar projects out there, yet none of them quite make the cut in terms of enterprise requirements.

• Ad-hoc connections


• ACL management


• User defined connections saved for later use


• Saved queries


• Multiple queries editing at once


• [ insert even more entreprise software mumbo jumbo here]

We're writing a Google Web Toolkit (GWT) front-end and a Spring based backend as a core. All data manipulations are possible thanks to the Olap4j API. There was talk of a Json bridge later on in development, but this requirement is not part of any sprint planning for now.

The project is hosted on Google code and all the project management is done in the Jira tracker. If you have any further questions about our project or want to chat for whatever reasons, we can be reached via the mailing list or ##pentaho.pat on freenode.

Data integration challenges tackled

Data integration in business environments can be a painful task. I mean REAL painful. The volume of data is huge, it does not cross-validate, it is dispersed in many heterogeneous formats, yadi yada. You know the song. Some day, I stumbled on Pentaho Data Integration (PDI).This was a real breakthrough.

First thing first, it's not subject to "vendor lock-in". It can read most data formats out there and can write it back to pretty much anything. This is a huge plus because gives it the ability to be used by a plenitude of user types and environments. Being written in Java also gives it an edge as an enterprise tool, for it is platform agnostic.

But the real advantages are not those trivial specifications. My love for PDI has much deeper roots. Simply put : it's powerful. Creating an integration process is a trivial matter. Drag and drop. Link. Execute. Those three simple steps will cover most of your business needs. Really, I mean it. Never again will I write a snippet of code to read a CSV file and write it's content in a database. Mark my words; NEVER! This is a waste of time and a developer who lives with his times should know that.

What about the real juicy stuff ?

As you suspected, there is much more to PDI than meets the eye. It can be clustered, it can use a database based repository for all processes, there are automatic documentation generation tools and is supported by a huge community. Many tutorials exist to address most business needs and challenges. It's well made, very stable and easily expandable with plugins for power users.

I strongly recommend to give it a try. The next version should be released soon and it will include many great new features. I met Matt Casters last June and had the chance to see for myself all the new functionalities that will make it to the next release. We're talking about visual performance bottleneck exploration and some more neat stuff you won't find anywhere else.

Cheers, and have a good time integrating !!

Pentaho on the iPhone

I successfully added the iPhone extension to my Pentaho platform today and I was more then impressed with the ease with which we can enable the whole platform to work seamlessly on those nifty little phones.

Oh yeah, I bought an iPhone too...

I'm slowly discovering the fun of having a cellular phone in my pocket. This is something that I never experienced before; never had a cell phone. I have to say that I'm glad it's a good phone, and sexy too.

The bottom line is : get one.

For those interested, here's the wiki page that says it all. Thanks to Will Gorman, senior developer at Pentaho,  who put this all up.