What is Amazon DMS
Every day, more and more companies are moving towards cloud computing, with...
Project Hop was announced at KCM19 back in November 2019. The first preview release is available since April, 10th. We’ve been posting about it on our social media accounts, but what exactly is Project Hop? Let’s explore the project in a bit more detail. In this post, we'll have a look at what Project Hop is, why the project was started and why know.bi wants to go all in on it.
As the project’s tagline says, Project Hop intends to explore the future of data integration. We take that quite literally. We’ve seen massive changes in the data processing landscape over the last decade (the rise and fall of the Hadoop ecosystem, just to name one). All of these changes need to be supported and integrated into your data engineering and data processing systems.
Apart from these purely technical challenges, the data processing life cycle has become a software life cycle. Robust and reliable data processing requires testing, a fast and flexible deployment process and a strict separation between data and metadata.
Project Hop wants to be your goto-tool for data processing. Our main goals are:
As long time Kettle (Pentaho Data Integration, or PDI) users, there’s a lot we’ve been able to do towards these goals with the Kettle platform. However, Kettle has a history of almost two decades, and a large installed customer base that requires stability and backward compatibility.
To build a platform that is ready for the next two decades, we need to look forward, not backwards. After long discussions with Matt Casters, the initial developer and former lead architect for the Kettle platform, we decided to leave the past behind and part ways with Kettle.
We used Kettle (release 18.104.22.168) as our starting point, but made some drastic changes:
Last but not least, we strongly believe open source software should be in the public domain. We still have some work to do, but intend to start the incubation process at the Apache Software Foundation and donate the Hop source code to the ASF sooner rather than later.
With Apache Hop, we are convinced that a lot of organizations can benefit from having an easy to use and powerful meta-driven platform.
Hop will never be finished. We’re getting closer to a first release, but there will always be more work to do.
Next on our roadmap are
There are many other long term plans we have in mind to further develop Hop that you’ll hear about in due course.
We invite you to join us on this journey and hope you’ll enjoy to see the project evolve and grow in terms of functionality and community.
Hop is a new platform, with a roadmap and future of its own. With the drastic architectural changes we made, we had to make the unavoidable choice of breaking compatibility.
This doesn’t mean we’ll ignore the existing Kettle community and user base who want to join us on our journey. We’ll start working on a migration tool that will allow you to import your existing Kettle/PDI code into Hop.
Know.bi has been engaged in tens of international Pentaho projects since 2012. Kettle/PDI has been the red line in all of these projects, and made up the lion's share of the work in almost all of these projects. Kettle is on of our most indispensable tools, but needed some solid refurbishing after 20 years of development. Project Hop is a lot more than a Kettle face lift. We want to use the solid Kettle foundations to build a new data integration platform that is ready to work with modern data platforms and is ready for the future.
Although Hans and Bart are actively involved in Project Hop's development, know.bi is and will remain a system integrator and team of consultants. We are convinced Project Hop will allow us to serve our customers better and faster, which is our ultimate goal as a solution provider.
Our existing Pentaho and Kettle service will continue for the foreseeable future. We consider a full switch to Project Hop in due time to be a logical and natural evolution.
We hope you're as excited as we are about Project Hop and we're looking forward to building successful data engineering solutions with you.