Today marks the one-year anniversary of general availability for Matillion Data Loader. It’s been exciting to watch the growth of our first SaaS product and its adoption by Matillion customers. You’ve moved more than 350 billion rows of data for free! We’ve come a long way in a year and are excited for what’s coming this year. More on that in future blog posts.
In the time since we first announced Matillion Data Loader, the pipeline business has grown and changed as well. To be frank, the data pipeline category has become commoditized. Connecting to a data source and loading that data into the cloud is no longer the unique province of one or two products. There are numerous startup and established companies who do data pipelines, and companies can take their pick.
Data loading products are obviously named because they do one thing very, very well. But there are nuances in how each product does the job, and for how much money. It’s important to look for the following criteria if you’re evaluating a data loading pipeline product.
First and most obvious, you need a pipeline product that integrates with your data sources. But don’t get that confused with a product that has the most data sources. There’s no need to invest in a product with a thousand connectors if you only need a dozen. What’s important is to find a product that’s going to get YOUR data into the cloud, whether that data resides in Salesforce, your on-premises SQL database, or another cloud data platform.
At this stage in the pipeline game, it’s basically a race to the bottom in terms of cost. Why would you pay a premium per connector when you can load your data for a nominal amount–or for free? Choose the product that aligns, that makes the most budget sense and reallocate the fees you would have formerly spent on pipelines to somewhere more useful.
In another time, pipeline maintenance was a chore. Changes in table selections and configurations meant rebuilding a pipeline from scratch. And even if you had editable pipelines, schema drift meant constantly monitoring and correcting pipelines to avoid data loading failures.
Today, you not only want a tool that has editable pipelines, but you want to choose a product with “self-healing” pipelines that can easily adapt to schema changes and propagate them to the destination, without assistance from IT or the need to code.
Many enterprise companies need support for a multi-cloud strategy, and multi-cloud will be a growing requirement for most companies in the years to come. Data can need to reside in different clouds and cloud data platforms because of different use cases and security requirements. You want to make sure that whatever data loading tool you choose is capable of loading data into the cloud, but also loading it from one cloud platform to another.
For many companies, migrating or regularly loading data into the cloud is the first part of the data journey. But it’s not the last. Getting data into the cloud is the easy part: making that data useful for analytics requires data transformation. If you’re not doing data transformation right now, you want to be sure that any pipeline tool offers an upgrade path to a more robust cloud ELT tool. This will ensure that once your data is in the cloud, you can use your cloud data platform to ensure that it yields powerful insights for your business.
If you’re looking for a data loading pipeline product that checks all of the boxes, Matillion Data Loader is a free, SaaS-based product that makes it easy to get your data into your cloud data platform. Matillion Data Loader features:
The post 5 Things to Look For in a Data Pipeline Tool appeared first on Matillion.
Leave your details below so we can help find the best solution for your organisation.
+44 (0) 203 574 8313
contact@analytium.co.uk
4th floor, 86-90 Paul Street, London, United Kingdom, EC2A 4NE
We use analytics to discover stories, patterns and trends in your data which will help you formulate ideas and create positive outcomes for the future. Leave your details below so we can help find the best solution for your organisation. We’ll call you.
