Matillion

Avantages

-Extremely easy deployment with cloud formation template.
-Excellent pre-authored integrations that are easily extensible/customizable.
-Easy to use UI to build complex transformations.
-Extremely flexible logic with static and grid variables to intelligently execute jobs and branch as needed.

Inconvénients

-No automated scheduling based on dependent table updates. (This is possible to set up, but not out of the box.)
-Support SLAs have gotten worse over time.

Alternatives envisagées 

Avantages

There is a vast number of options and data connectors on offer.

Inconvénients

When communicating with the support and billing teams, they are never around to answer. Phone support is useless as no one answers, and it takes a few days to get an answer via email. Took for ever to switch billing from credit card to invoicing.

Avantages

Early on in the company, our non technical stakeholders could learn how to set up transformations on Matillion and build them up themselves. It is good to see the transformations visually.

Inconvénients

If you can write SQL and schedule them using DBT, you should do that. Trying to set up the same transformation that you write in SQL in 15 minutes will take you 60 mins with Matillion because you have to think differently when using a click-click software like this.

Avantages

Matillion is a cloud based ELT tool and it is very easy to use . The data transfer rates are incredibly good since the product locates in cloud infrastructure and the infrastructure can be scaled up when required.

Inconvénients

The connectors to other data sources are limited when compare to what other rival product are offering . The job monitoring functionality is limited , the scheduling and job log features could have been improved. The GitHub integration could have been straight forward.

Avantages

- Very intuitive design environment - super quick to learn.
- Components map closely to SQL capabilities in Redshift.
- Easy to view the SQL generated.
- Sample, row count, and data lineage make it easy to unit test as you build.
- Powerful management options via REST API (orchestration, Github integration, etc.).
- Native AWS integration.
- Aggressive release cycle, constantly adding new features.
- Easy to document in-workflow.
- Lots of third party integrations.

Inconvénients

- User concurrency/licencing model (simple, but can't be customised to fit your need).
- Exception handling could be improved.
- A single Task View would be helpful for multi-environment/project support.

Alternatives envisagées 

Pourquoi passer à Matillion

Avantages

Web-based, Runs in AWS so it can sit close to your data if you're using a cloud warehouse. You only have to pay per minute that you have the instance on which is a BIG plus.

Inconvénients

Not nearly as many features and user-friendly data transformation tools as a competing software such as Alteryx has. As such, this tool is not as convenient or flexible for detailed data transformations. We mostly use it for bulk loading which it is very fast and good at.

Alternatives envisagées 

Pourquoi choisir Matillion 

Logiciel antérieur 

Pourquoi passer à Matillion

Avantages

It is extremely fast and easy to pull data from various sources and pipe it into Snowflake. The graphic interface lets this happen without programming.

Inconvénients

There is almost no ability to get data back from your warehouse into the other systems to synchronize the data. It's great to have it all in the warehouse, but it seems pretty critical to have that data flowing back to other systems that are part of your environment.

The pricing model is frustrating. You are billed for when the machine instance is ON, not when it is actively doing something. So, if you have a couple hours of job that run in a day, you have to shut down the machine to save money the rest of the day. With it shut down, development can't happen. So, we had to turn it on and off all the time. It would be much better if they billed as it is used, not as it is on.

Avantages

Matillion is incredibly easy to deploy in AWS. In addition, the security and compliance aspects are all handled on the customer side due to the deployment method.

The UI is easy to use and the price point was the most competitive.

Inconvénients

We haven't used it enough to encounter negative aspects yet but having to pay for the full amount of time the VM is up is going to be cost prohibitive for our small team.

Alternatives envisagées 

Pourquoi passer à Matillion

Avantages

The collection of components; especially the integration components that hook into a very popular CRM and another quite popular ERP.

Inconvénients

The python component. Depending on the box where your instance is hosted, this component will drag along like a broken train or move at the speed of shooting star.

Avantages

It is a pay for what you use type of software. Most other ETL platforms are a flat monthly fee, sometimes at crazy prices. The pay for what you use model makes it easy to justify cost to leadership. You can schedule jobs to make sure you maximize your time.

Inconvénients

The user experience feels a little dated. Some things could be better displayed/ explained.

Alternatives envisagées 

Pourquoi passer à Matillion

Avantages

- Provides great framework for visualizing complex queries with business users that are not SQL savvy
- Logical flow of Transformation builder allowed quick transformation from file based to database based ELT
- Just works! We have had very few issues with the tool over the 2 years I have been using it.

Inconvénients

- Would like a more robust scheduler
- Would like built-in messaging (e.g. for job failures/success) to be able to email out. This is not provided in the version for Big Query that we are running on GCS.
- Better source code control (at the transformation/orchestration level, rather than the whole project)
- Always could use additional documentation in how to use some of the more complex parts of Matillion.

Avantages

1) Most of the components are user friendly.
2) Development of ETL orchestrations and transformation consumes less time.
3) Advanced features are available in some of the components makes the complex scenarios achievable.

Inconvénients

1) OAuth document does not provide the details of proper permissions and access levels on account_id or client_id which are required for a connector.
2) Differentiation of naming convention of metrics between the console and matillion data model , mapping document of these naming conventions is not available.
3) Connector for Outbrain is not available.
4) Indirect file loading concept is not available , for example if we have five files with same structure , reg ex can not be used to load all files in a single component(s3 load,s3 put or excel)
and need to use the file iterator.
5) Loss of properties when the component configuration is changed from Basic to Advanced features , ideally the component should include all the properties of basic and then additional setting should be provided.

Avantages

- Easy to orchestrate an ETL pipeline
- Integration with Redshift and other AWS services provides convenience
- Support team is easy to reach and have relatively quick response times
- They also support the other 2 popular cloud data warehouses, Snowflake and BigQuery
- Some components are available out to the box for easy integration
- Task history is useful for debugging and monitoring current state of ETL pipelines
- A GUI is provided for implementing complicated workflows, that can be easily followed

Inconvénients

- Since it's mainly GUI based, it becomes difficult for data engineers to compare changes between old and new versions of an ETL.
- There are hard limitations for the number of concurrent users. This is part of the pricing model which is the reason.
- Integration with GIT is not yet available but we heard that it would be coming in the near future.

Avantages

Matillion is very intuitive.
The GUI is easy to use and fairly easy to learn.
The support team is great and they respond to your questions quickly.
I like how the Matillion team is frequently looking to add new functionality and features. The new keyword search functionality is very helpful.

Inconvénients

Matillion works great extracting data from a variety of databases and is very efficient unloading onto S3 or Amazon Redshift. However, when we attempted to use it to unload large datasets to Microsoft SQL Server it was very slow and we experienced performance issues.

Avantages

It is easy to use and quick to get started, there are many inbuilt APIs and transformations that integrate it across a broad suite of systems, and enable you to transform data for the business without having to spend too much time thinking about scripts or bespoke batch files.

Inconvénients

The filter transformer is pretty minimal considering the power of the SQL it is utilising, it cannot use compound logic or reference other columns/objects in the query.
There is also optimisation issues in very complex workspaces.
Both of these have easy to use workarounds but it is unfortunate it is not supported more at surface level.

Avantages

It is very easy to implement features and changes to existing workload. It is very good for creating POCs

Inconvénients

It is not scalable and has numerous limitations. Concurrency setting for Matillion limits the possibility of using it for multiple warehouses. It is highly insufficient when executing commands parallel and does not utilise the resources to it's full potential.
The environment limitation is also very problematic when trying to scale.

Avantages

My background is development and I love the simplicity of the Matillion, you can very easy transform your thought process into blocks in Matillion, and debug any issues at a very granular level.

Inconvénients

Matillion crashes from time to time, meaning you need to log back on to the system, but its benefits definitely outweigh this small issue.

Avantages

Its very easy to learn. The UI is great and easy to navigate. It utilized the ELT approach which makes the data integration fast. Security and access control is well implemented and there are a lot of components to use.

Inconvénients

Its a one way ETL, I'm using the BigQuery product and all the destination are in BigQuery, it would also be nice if we can load data to the sources.

Avantages

Simple, intuitive interface. I replaced a complicated series of scripts, "ETL" products, and Python code with a simple-to-understand Matillion instance.

Inconvénients

We use it on Snowflake for Azure, so there are several features that have not been migrated from the AWS version, yet.

An option for a larger server by default in the Azure Marketplace.

Avantages

User interface and range of connections.
Powerful capabilities.
Compatibility with AWS

Inconvénients

Licensing is not flexible, Why cant you specify the number of concurrent users for a given instance? It comes with a fixed amount, if you don't like it, update to a much large and expensive instance that we do not need.

Avantages

Connectivity
Really good customer service
Individual solutions if necessary
No downtimes
Hosted on Amazon

Inconvénients

Pretty expensive
A bit hard to set up initially

Avantages

very easy to use, fast and good Customer Support, many API Connectors out of the box, well integrated in AWS Infrastructure

Inconvénients

some Problems regarding Out of Memory lags, sometimes it's better to write custom Code instead of using the Out of the Box API Connectors (faster and more stable)

Avantages

This tool helps in automating etl process.

Inconvénients

Integration with python is limited. Need this feature

Avantages

Connectivity to Saas sources. Reliability and Customer Support team

Inconvénients

RSD creation for Saas sources that do not have built in connector.

Avantages

Inbuild features to integrate with AWS s3, send SNS messages.

Inconvénients

Readability should be more, ease to retain changes or navigate to environment variables should improve.