A bit of fun for a Sunday.  Below is the list of my top 10 interesting companies in Data Management right now.  Interesting to me means doing new stuff and being somewhat disruptive, or have a “watch and see” quality about them.  Note this is about companies not data management applications. 

While I find a bunch of other data management applications interesting (PNUTS, Cassandra, Redis etc) these aren’t really encapsulated in a company with a go to market strategy.

10gen – They are making interesting noises not sure about delivery yet
Amazon – SimpleDB is neat, but not a grown up data platform yet
Aster Data – Doing funky things with Map/Reduce
GroovyChannel – Are they nuts, they have to change the name of GroovyDB?! *
Kickfire – I love this appliance
The Monty AB Program – Is Maria the new “peoples” MySQL
MySQL  – Because who knows how the Oracle merger will pan out
PostgreSQL Global Development Group – Or is PostreSQL the new “peoples” MySQL?
Wistla – in stealth (shhhhhhh)

Ok you are probably thinking that’s only 9.  I will take feedback in the comments below to help me choose #10 as I am currently undecided on this one.

* Update: GroovyChannel is now know as Groovy Corporation and GroovyDB is known as SQL Switch.

A great post from Jonathan Ellis on "Why you won't be building your killer app on a distributed hash table"

Scroll down to see my comment.  I agree a simple DHT is not a suitable solution for mainstream data management issues, but also I think that there is a gap between RDBMS and DHT.

I met with a friend of mine in New York recently who runs a credit card processing end-point company.  They specifically built their business around a non-relational database platform and feel they would have major issues had they chosen to build their business around a traditional relational database.

In this business they process hundreds of transactions a second, this means that hundreds of people every second are standing somewhere in the world at a checkout making a purchase.  And if the transaction is delayed, that is a lot of people are standing and waiting.  Worst still if the transaction is delayed too long the transaction fails and the customer not only has to retry their transaction, they are likely embarrassed and probably somewhat annoyed.

The relational database certainly has the capability to scale to many hundreds of transactions a second. But the reasons for their move away from the relational database aren’t specifically scale related but instead come down to predictability and consistency.

Relational databases are not predictable or reliable in terms of consistent performance for a number of reasons.

Firstly, every query uses a different amount of resources.  A query could consume 1 or 2 I/O’s or 1 or 2 million I/O’s depending on how the query is written, what data is selected and factors such as how the database is indexed.  Performance is further varied by how the database is maintained (fragmentation).  What makes matters more complex is that different predicate values for a query can have vastly given data distributions.  The same query executed with different constants can have vastly different resource requirements.

Because every query has a different “footprint” running a query in isolation does not provide indicative statistics on how that query will perform under concurrent load.  In fact is become impossible to predict the exact execution duration of a relational database query as its performance will be dependent on what else is being executed at the exact moment it is.

Cost Based Optimization

Because queries can have complex execution steps behind them with multiple paths to achieving the result, one neat thing that was added to the relational database was cost based optimization.  This takes the control of “how” a query is resolved away from the developer and instead the relational database server works this out at runtime using a complex series of algorithms.  This was hugely important for simplifying the use of relational databases.  But it is bad for predictability.

Cost based optimization can get it wrong.  It is much less common now than 10 years ago but every DBA has still come across cases when the optimizer chooses the wrong “query plan” and the query execution takes a long time for seemingly for no reason.  But worse for predictability is cost based optimization is dynamic and looks at a lot of factors at run time to determine which plan is best.  These factors can change meaning that query performance can change (often without any external schema/code modifications).  This dynamic ability can cause headaches to DBAs who are often tasked with resolving performance issues when suddenly the database is “going slow”.

Example:

Ok let’s take a real world look at this problem.  I have gathered some stats from a real, production (SQL Server) relational database.  This database is around 500GB in size and processes ~700 transactions per second 24 x 7.  This data is just a couple of minutes of captured activity.

The first graph I am showing here is the # of queries based on resource usage.  To simplify things resource usage in my example = CPU TIME + # READS + # WRITES. 

Because queries in relational databases can have vastly different resource usage profiles, we get a very wide distribution of resource usage by query (note the logarithmic scale).
 

We can see that most of the queries executing on this database use <50 “resources” (CPU + DISK) but at the same time a smaller number of queries can use anything up to 22750 x this.  That is a massive distribution. 

Problems with predictability can start to occur when a number of high resource queries randomly end up running concurrently.  This restricts resources available to other queries and therefore the execution times of other queries execute at that time are affected.

We can see this by looking at the execution duration of ONLY the queries that have used 50 resources (CPU+DISK) or less.  They have a similar resource usage profile, so in theory they should all have a similar execution duration, right?
 

Unfortunately no.  This graph is showing the execution times for only the queries that used 50 resources or less.  While we can see that most of these queries executed in 100ms or less, many of these queries took a lot longer than this.  In fact there was a wide distribution up to 5297 x this base execution time (529 seconds).  All these queries used roughly the same amount of server resources yet their execution times were massively different.

The reasons for this could be many.  Firstly it could be resource contention.  Many of the queries taking longer could be doing so because they are being executed at the same time as a number of high resource queries.  Or alternatively, it could be because of resource concurrency issues.  These queries may be delayed because they are waiting for blocking queries to complete.  Or it could be because of internal database management issues (indexes updating), or it could be due to concurrent disk issues (log backups, check-pointing) and so on and so on.

These are in fact all issues that DBAs deal with and investigate on a daily basis.  The best you can do is to try and optimize as many of the reoccurring queries as possible.  However in doing this it still isn’t possible to accurately predict performance at run time, or ensure consistency of execution time at run time.  This is bad news when you are trying to use the relational database as a platform for applications where consistency of performance is paramount.

This is the first detail post in a series I am doing focusing on the issues that exist today with the Relational Database.  This first post is on the deployment model.  It could be argued that this isn’t directly related to the “relational database” but rather is an implementation model problem.  I disagree with this as many characteristics of the relational database lead to the deployment model described (we will explore in later posts).

For most of my career I have been involved with the enterprise and the databases in this environment.  Over the years I have seen the volume of databases increase dramatically in line with an increase of data centric applications.  This has led to even medium sized organizations often having dozens of physical database servers.   Enterprise organizations often have hundreds of database servers, occasionally thousands of them.  The volume does vary heavily by database platform however, SQL Server typically suffering the most sprawl out of all the mainstream enterprise relational database platforms.

Problems happen when DBA’s try to co-locate independant databases on a single server.  The problems are due to the dynamic nature of databases in terms of data volume and dynamic nature of query load.  This dynamic nature makes managing capacity a complicated and time consuming task.  When relational databases share resources you risk a small number of intensive database queries causing concurrent impact to a wider group of other queries.  Because of this, typically small numbers of databases share the same servers.  On average for SQL Server around a 10:1 database to server ratio is seen in the enterprise.

The brokenness of this model is pretty easy to spot.  Firstly, resource inefficiency and ineffective distribution is a clear problem.  While I am generalizing somewhat, an organization with 100 database servers often could have 70% of those servers vastly underutilized, 20% of those servers effectively used and 10% of those servers highly over utilized with users suffering from poor performance, “bottlenecks”, as a result.

With this deployment model it isn’t possible to take the unused “resources” (CPU, Memory, I/O bandwidth) from elsewhere in the organization and re-apply it to where needed (even with downtime, let alone in real time).  Instead new infrastructure investment is made to continually add new resource capacity for the bottlenecked databases.

A relational database is capped by the limits of the server on which it currently sits.  A DBA monitors the server trying to keep current query demands as optimal as possible to avoid premature bottlenecking, and continually planning to stay one step ahead of database requirements growth.  This is a costly process and one often not helped by the unpredictability of the relational database (which we will discuss later).  Multiply this need across the hundreds of servers described and you can imagine it is a significant contributor of the cost of ownership.

When you reach the limits possible on a single server many database platforms have few practical options available for further scalability (such as distributed scalability for reasons again we will address in a later post in this series).  Too often organizations with multi-million $ servers are being forced to split workloads, move real time operations to batch operations, replicate data for offline processing purposes and mandate specific times when users can run particular intensive functions.  Again, all this manual fiddling becomes a management nightmare and significant overhead when you multiple it out.

This issue in isolation can potentially be addressed through technologies such as virtualization.  While virtualization is yet to make major impact on the way in which production databases are deployed in the enterprise, this may change in the future.  However as we delve further into the problems associated with the relational database, we will see this is not the only issue that we face taking this technology forward.

The relational database has been the core mechanism for structured data storage and retrieval for the past 30 years.  My career so far has focused around the relational database, whether it be from a development, administrator or investment perspective.  In all this time the RDB has been the best generic option available for developers building data centric applications.  The generic nature of the RDB has made it suitable for wide mix of application requirements, be they heavily transaction processing orientated or heavily data analytics related.

However over the few years we have been witnessing a slow shift aware from the “RDBMS” for everything trend that we saw over the preceding decade.  And this is occurring because the demands we are placing on data in terms of scale and volume are growing to a point where the most generic platform is underperforming and instead more specialist database technologies are starting to be selected based on their closer fit with the requirement.

This trend has started in and is therefore more visible in the data analytics space.  The specialist solutions have be slowly cropping up over the last 5 years and now today it wouldn’t be that unusual for an organization to choose a specialist data analytics database platform (such as those offered from Netezza, Greenplum, Vertica, Aster Data or Kickfire) over a generic database platform offered by IBM, Microsoft, Oracle or Sun for housing data for high end analytics.

My argument is that while I see the traditional generic RDBMS remaining the platform of choice for most generic application requirements in the foreseeable future two breakaway alternative paths are also emerging.  The first is that I mentioned above, a reduction in the generic aspects of the RDBMS with a specific focus on high end data analytics functionality.  The second, which I see starting to emerge right now, is the opposite of this.  A reduction in the generic nature of the RDBMS with a focus on the specific requirements of high end transaction processing.

Starting tomorrow I will be presenting a series of posts that discuss real world issues facing the RDBMS when used in transaction processing environments that are being encountered today to highlight why this alternative path in transaction processing is appearing then following this I will present a series of posts on the technology that is emerging in an attempt to address these weaknesses.

I had the pleasure of talking with Jay Kreps yesterday about Project Voldemort, here some notes from this conversation.

• Jay is a cool guy, practically bounces off the walls when talking about Project Voldemort.  I think I am like that sometimes when talking about things I am working on.
• Project Voldemort is the open development of something that was created to serve a need that LinkedIn had.  LinkedIn is supporting the project and supporting the move to make it open.
• There is no apparent short term plan to turn Project Voldemort into anything large, such as a spin-off company or a cloud based service.
• Voldemort is used extensively at LinkedIn for online queries.  Volemort is not used for analytical queries, data is instead fed into a data warehouse for this purpose.
• The primary focus of Project Voledmort is scalability and predictability of performance.  Voledmort is a key/value store, not document orientated storage.
• Project Voldemort uses a hashing algorithm to allow any node to know that location of data on any other node.  This avoids the need to have centralized lookup nodes (which results in multiple hops and potential bottlenecks).
• Project Voldemort doesn’t use REST.  During testing a REST interface was found to have too great a latency and instead Project Voldemort has its own native driver interface.
• The name of course comes from Harry Potter.  For those not familiar, Voldemort’s sole was broken up in pieces and to kill Voldemort Harry had to kill all the pieces.  Project Voldemort aims to mimic the spirit of this, to have scalability but with no single point of failure.

As I have blogged about before, I love companies that re-invent themselves to stay current rather than try to wait out market changes as if they were temporary detours.  This is why I love Amazon, who would have picked the world’s largest book store would become the world’s largest webservices provider, even only a few years back.

In a recent article for Read Write Web I wrote about how Amazon’s SimpleDB is a cost effective and scalable data store, albeit with certain major limitations.  And as I indicated in this article, many of the key/value data store vendors were taking the approach of using simplicity of functionality as the means for getting the scalability problem solved.  Then working backwards, re-introducing features over and above the basic CRUD operations when they had solved their respective scalability impact.

Last week Amazon added to SimpleDB, rolling out a couple of features that puts another brick in the bridge between relational and the current key/value stores.  Amazon added COUNT(*) to their SQL stimulant but more importantly appear to have added the ability to run queries that take more than 5 seconds.

While their post incorrectly says that the NEXTTOKEN feature is documented in the release notes (it isn’t), essentially it seems they have added a tokening system which allows a developer to recursively call the execute request for a complex query.  On each request SimpleDB will carry on from where the previously request left off until eventually the query completes. 

While on its own this is just a baby step forward, it is important as it shows Amazon understands that even the most basic of applications will have some form of analytical requirement.  The need to run complex queries when you have large datasets is important even for online transactional applications.

While not yet a direct threat to the relational vendors long term stronghold on the database market, 5 years out if such constant reiterations occur, locating data in the cloud could become a serious and attractive option for much more than bleeding edge web apps.  Microsoft realizes and is in the game with SQL Data Services, it will be interesting to see how the other vendors (namely Oracle and IBM) respond.

Last week I spent some time talking to Michael “Monty“ Widenius (co-founder of MySQL) about his decision to leave Sun.  The bulk of the forward looking aspects of this conversation are going into a post I am preparing for Read Write Web.  This should be out sometime next week. 

While I didn’t want to focus what happened in the past too much in this article I thought I would post some of my own personal assessments I took from my conversation with Monty here.  They are as follows:

• As almost everyone who meets Michael concludes, Michael is a nice guy.  He is transparently open and a pure believer of a common good.  He epitomizes the true meaning of open source, which is much more than a licensing model but is instead a methodology for a shared collective.
• MySQL started life as a community developed project, but since 2001 this has steadily decrease to the point where most MySQL development now happens at Sun/MySQL.
• This has not been something Michael has supported, over time he has pushed harder and harder for the community focus to return to MySQL.  If Sun or MySQL management (Michael was not the CEO) shared this vision or not is unclear, however I think Michael felt he was being constantly delayed from addressing this.
• I think Michael believes the long term future of MySQL is dependent on regaining MySQL community involvement.  I think he felt so strongly about this that he believed he had to leave Sun to achieve this.
• I believe Michael left Sun because he felt the long term future of MySQL (his baby) was at risk if he didn’t take action to address some of his key concerns.
• His public criticism of the release of MySQL 5.1 with “issues” still present appears to have been the straw that broke the camel’s back.
• It seems some of the MySQL employees didn’t personally do as well as liked out of the MySQL/Sun merger.  In Michael’s vision for how a company should operate, all employees should share in the success of a company.
• I think much of what Michael believes to be the “right way” to do things was always going to be somewhat incompatible with a traditional large company, especially a traditional large public company.
• A few days after Michael announced he was leaving Sun, Marten Mickos (the MySQL CEO) also announced he was leaving Sun.  Less than a year on the integration of MySQL into Sun appears to be only part done, how this integration continues I think is now a new concern.
• It has been reported that a number of employees (especially those working on Maria) are likely to follow Michael to his new company.  This appears to be accurate.
• Some media outlets also reported that a number of employees were also likely to follow Marten.  This seems to be confusion in their reporting, and is instead unlikely.

As I mentioned the forward looking aspects of the future of MySQL will be addressed in a subsequent post.

The music industries decision to do away with DRM, which has lead to Apple’s ability to offer titles DRM free in iTunes is a great decision that will encourage people to purchase more not less of their product. 

Time and time again we are seeing examples of these brute force approaches to content protection fail.  Regionally encoding DVD’s was/is a terrible idea, however largely irrelevant as most DVD players are multi region.  DRM was a long suffering headache for the digital music revolution that was doing more to hurt the sales of digital music than protect the artists.  There are still examples when people are trying to protect content using old world mentality in a new world medium, such as the BBC iPlayer which tries to restrict viewing of their published shows by geographic location (and similar services exist in the US).
To date content protection has all been at the cost of convenience, freedom and flexibly of the legitimate user.  If I am travelling to the US and see a DVD I like, why shouldn’t I be able to legally purchase it and watch it when I get home?  If I buy a song on iTunes, why shouldn’t I be able to decide to listen to it on my Blackberry or other device I happen to choose at the time?  If I watch BBC TV online, why can’t I continue to do so while in the US on business?

However content protection is a valid pursuit which isn’t just about monopolies trying to protect their turf, it is about protecting ones digital assets whether than be commercial assets or not.  The field of content protection will evolve dramatically over the next decade.  And when we get it right content protection will be something useful for all of us and something we don’t really think about on a daily basis.

Some key hurdles that we will:

• Content protection will have to stop being a standalone fence around content and become part of the content itself.  The problem with creating barriers around content is that it only takes one person to move that content outside of those barriers, then those barriers no longer apply and the content is no longer protected.  Instead having the protection as part of the content ensures the relevant protection mechanisms are always in place.  DRM was an attempt at this but it was a method that put the needs of the user last and failed because of this.  A common standard for all forms of content will have to be agreed upon.

• Authentication & Authorization will need to be fixed first.  Today there is no single standard for determining who you are across multiple platforms and devices.  Before content protection can work we need to ensure there is a common standard in place that allows you to be you regardless of where/how you are accessing content.  The content producers won’t own or control this, in fact no single authority will control this but a common standard will fix the authentication mess which exists in the digital world today.

• Content protection can and will be policy based.  You can define a specific authorization list or you can define demographic attributes (such as age restrictions) which allow you to control who can access that content.  It is not just about granting content based on named users.  You can grant content based on age, relationship to you (friend, family, associate), employer etc.

So imagine if you publish a photo of your family you might choose to limit the content to only be viewed by members of your family.  It doesn’t matter who copies or re-publishes that photo because the content protection mechanism is built in and only people who are authenticated as a member of your family can view.  Maybe you publish adult material (photos, videos etc), you can assign a content protection mechanism that ensures only people 18 years or older can view the content (regardless of where it is published or redistributed).  Finally if you are a musician you can assign a content protection policy to your music which specifies that only people who have a licenses to listen to your music can do so.  Then it doesn’t matter who sells the physical bits, what format it is in, what device the music is played on.  People don’t even need to download the music they own, they could go to any jukebox web service and only see the tittles they are authorized to play.

Clearly there are numerous challenges to face until we get to this point, but until we do any content protection mechanism is just going to be a facade causing more inconvenience to authorized users than it does to unauthorized users.

Oh Twitter, Twitter, Twitter.  In almost every respect Twitter has been a perfect example of how not to launch a start up.  They have made blunder after blunder and in application of conventional logic, they shouldn’t be around today.

Issues that come to mind (that I remember) include upsetting half the world when they one day cut off the TXT msg service to everyone but the US & India (without any prior notice) and removing the instant messaging feature (without prior notice).  Then months of availability problems (the “fail whale”, how many other successful start ups have a common term for failure in their users vernacular?).  They have been extremely slow to roll out any new features, I can’t think of any significant features in the last 12 months, their web interface is outdated when compared with third parties.  Yes they have an API for integrating but they have on occasion changed this without warning breaking some third existing party applications.  Now it is security, a whole bunch of users passwords were phished over the weekend and following on from that a number of high profile “celebrity” accounts were hacked in a seemly separate incident (apparently hacking twitters support tools, whatever that means).

But does this matter?  I am still using Twitter today, and I am sure Twitters own numbers while show their user base has remained similar this week to last week.  Maybe it has even grown due to the publicity.  So this begs the question, is love really blind?  Twitter provides a micro community which you can’t really get anywhere else and the users love Twitter because of that.  And just like many relationships this love gives Twitter some freedom to screw up and have their users forgive and forget time and time again.  But also just like being in a relationship this love is not necessarily forever.  If you screw up big time there may be no forgiveness.  Twitter hasn’t done this yet, but I think they would find it hard to recover from a multi day outage or mass data loss for example.  And of course the other potential cause of love loss is something much better comes along.  A more attractive, more dependable alternative that has their head on straight and doesn’t screw up all the time.  But because people are so committed to Twitter this is unlikely to happen unless both of these scenarios happened at the same time, but for Twitters sake I hope it is able to get itself together, grows up a little and makes their users proud.