I wanted to be very open about how I write my reviews, and what information I use…
Updated 22 Jun 2017 22:01 UK Time: To reflect that I may choose to share reviews before publication if it helps improve the content and accuracy only. Changes underlined.
Currently my reviews are literally the last release of the product, and maybe a couple of point releases before. This means any great new strides will be included.
Given I’ve written a couple of books on NoSQL I obviously know a lot about these products already. Therefore any good and bad points that are generally true of the product (E.g. no ACID support, good query support, scaling out to hundreds not just tens of servers) will be generally mentioned in passing too.
I do not yet have a section for ‘everything about database X’ – although in future I aim to do a similar thing to how MacRumors keeps their ‘Macbook Pro’ page updated. See here for a sample: https://www.macrumors.com/roundup/macbook-pro/
This will come in future versions though, after I’ve got a couple of versions of reviews for each database.
Which databases I review are decided in a couple of ways:-
- A request from the community, usually via a public vote on twitter
- I’ve reviewed them before in NoSQL for Dummies or State of NoSQL 2016, and want to ensure the current information is ‘out there’ somewhere
- I’ve seen an exciting development in their software or company on the news, or more likely on twitter
- If I get bored, I may look at the db engines list of databases and start with ‘most popular’ and work my way down. I don’t trust the metrics on this site though, so don’t rely on it.
I have a top level summary of each product, with links to the company, press release, and detailed blog post(s).
Below this I have a What’s new section aimed at people who are familiar with the database, but who want a very quick 5 minute update.
Then I have the Things I Like and Things I don’t Like sections. These are entirely subjective and are my own personal opinions. It should be said though that for my entire professional career I’ve worked on mission critical Enterprise systems in Financial Services, Insurance, Central Government, and Defence. So Enterprise features are a must and will generate many effusive positive comments.
I also have a few bug bears which I’ll explain in future posts. Having any of these issues will result in some brief negative remarks (I don’t rant though):-
- Databases that are useless unless you pay for the Enterprise version
- Use of the AGPL ‘open source’ (bah!) license which stops people using the community edition on for-profit projects
- Lack of ACID compliance
- Lack of search or query functionality
- Lack of security
- Over inflated marketing claims (E.g. ‘Webscale’, or ‘yes we are multi-model!’ when you’re not really)
- Strong Consistency (actually means ‘no consistency’), or other marketecture weasel words
- ‘New’ databases which are just re-hashes of established products
- Bull$^!t company valuations based on private total annual revenues that assume software company flotation level returns, when in fact 75% of your revenues are from consulting, not software licensing (There’s a lot of this going on in NoSQL land)
I do contact the vendors for permission to use their logos, and to ask for information that is missing from their website prior to publication. They do not routinely get pre-publication copies of the posts though. This is important for editorial freedom.
I may choose to share a review before publication, however, if I think their website has out of date information so that they have the opportunity to publish updates before my review is published. I will only give a few days for this though, otherwise the review gets posted anyway as-is. Having worked in software sales for a number of years, I sympathise with trying to keep product material fully up to date! I like to give people a fighting chance on the assumption we’re all trying to make a better, NoSQL filled, world!
Having said that, after publication if they spot and error and point it out, I do promise to alter the review to reflect this as quickly as practicable, usually within a week. (I have a day job, after all, not to mention holidays!)
All updated reviews are marked as such, with a brief overview of the change at the top of the article, and the sections that are changed underlined. They are also publicised to twitter in the same way as the original article was.
To research, I read the latest press release, then blog post (to get the real answers!), then ferret around technical documentation, blog posts, and complaints/praises by others on the web. I trust things on the web with a pinch of salt though, so I always research any problems mentioned in vendor documentation to confirm there is in fact an issue. No ‘fake news’ rumours, therefore, on my reviews!
I hope that explains the review process. If you have any additions you’d like, please tweet me @adamfowleruk !
Adam's Deep Technology Blog 