Just because organizations have been diligent about collecting data doesn’t mean that they have been properly leveraging it. Each organization has its own distinctive data sources that pull in information on numerous users and accounts. Data ecosystems keep getting more complex, and the amount of different data sources within a company only continues to get more vast. With the help of cloud data warehouses, consolidating data into a single location in order to get rid of data silos between different sources is important in this data integration process, which is what the tool called Matillion is for.
Here are the key things you need to know about Matillion before giving it your consideration.
1. Matillion Explained
Matillion makes a claim to be an all-in-one SaaS system and platform for cloud data integration that handles the whole process from acquisition and ingestion to transformation.
What Matillion users receive is a low-code solution that makes use of the computing efforts of cloud data warehouse systems, leveraging your cloud environment to customize your data to meet your team’s needs.
The more products under the Matillion umbrella are Matillion ETL, their durable ETL solution, and Matillion Data Loader that is used to move data around, with both offering a thorough degree of arrangement for advanced users.
2. Matillion Data Loader
Matillion Data Loader is a free tool that pulls out data from your source systems and loads it into the location of your selection.
Being that Matillion doesn’t charge anything for Matillion Data Loader, it doesn’t come with many features, such as the fact that you can’t perform any transformations.
Matillion Data Loader only supports approximately 35 connectors, and when it comes to destinations, it doesn’t offer any support for Azure Synapse or Delta Lake.
This point-to-point tool for transporting data provides a simple wizard to construct data pipelines for extracting and loading your data. But it doesn’t provide any automation capabilities or additional developer tools.
3. Data Integration with Matillion
Data integration is the procedure for consolidating data from separate data sources into a single unified view with the ability to derive insights from it.
The main purpose of data integration is to keep data democratized and get rid of data silos. Data integration can be broken two into three key parts, which are data acquisition, data ingestion, and data transformation/preparation:
- Data Acquisition: The location where data is initially collected (Salesforce, Marketo)
- Data Ingestion: The method of moving data from different sources or operational systems to
a new area, such as a data warehouse (Snowflake, BigQuery).
- Data Transformation/Preparation: The point at which raw data is transformed and prepped
for analysis so that analysts can leverage business intelligence tools like (Tableau, Looker),
etc., to make comprehensive reports and dashboards.
4. Issues to Keep in Mind
Matillion has positioned itself as a SaaS solution, but in reality, Matillion could be categorized as an iPaaS (integration Platform as a Service).
It gives you a platform for data integration, but the user has to handle all of the extra details and set it all up.
Matillion is an ELT platform that should really be seen as a data orchestration platform because Matillion actually allows you to coordinate monitoring and execution of the data pipelines and workflows that you rely on.
This way of doing things creates problems and won’t scale particularly well. In fact, setting up these data pipelines creates difficult workflows, depending on your data ecosystem.
You will have to address issues when they arise. ETL and ELT solutions are supposed to help to make an engineer’s job easier and give them some spare time, not having to maintain pipelines without writing code.
Also, you should be aware that Matillion largely serves enterprise customers, doesn’t put much emphasis on SMB/mid-market businesses, and doesn’t provide much support for smaller companies.
5. Issues and Reverse ETL
Reverse ETL is the method of copying data from the data warehouse to operational systems of record, such as SaaS tools used for growth, support, marketing, and sales.
The data warehouse needs to be your organization’s standard of truth, and the information that it holds should be accessible to your other internal teams so they can leverage it for the company’s success on a regular basis.
Unfortunately, even though Matillion specializes in ELT, you can’t use Matillion for Reverse ETL.
In the end, Matillion is a data integration tool, but to be clear, it isn’t the industry standard. One of the reasons is because it adds to the workload of your team that places a heavy amount of work on your team.
Discussion about this post