Elevating marketing intelligence with Azure cloud solutions

CIGen partnered with a US-based marketing client to enhance their Big Data processing capabilities using Azure, .NET, BLOB Storage, HD Insight, Apache Spark, Data Lake, and ClickHouse.

This collaboration focused on migrating from on-premises to the cloud and from SQL-based services to Big Data technologies, optimizing the processing and distribution of marketing datasets for SMBs.

Ask a question

Tech stack

Azure

BLOB Storage

Apache Spark

.NET

HD Insight

Data Lake

Clickhouse

About
our client

Location

USA

Industry

MarTech

Timeline

In this case, the client provides IT services for its clients as well as perfects the use of Big Data to do its goals. The company targets small and medium-sized businesses (SMBs) that generate data volumes of up to hundreds of thousands every day and uses these volumes to create valuable marketing/advertising insights.

‍

The challenge

Our team joined the web application when it was already an MVP. The main task was to carry out the migration to the cloud, which took place on 2 levels.

Initially, we aimed to migrate from on-premises to the cloud, since hundreds of millions of data pass through the system every day, and it would be problematic to increase the amount of hardware for such conditions with a classical approach.

The second migration was from SQL-based services to technological ones related to big data. This is, from a database project to a cloud project since new modern technologies require modern solutions. These main tasks were our top priorities.

‍

Collaboration goal

To migrate and optimize the client's marketing dataset processing platform using advanced Big Data and cloud technologies.

Services we delivered

Azure migration

DevOps

Data analytics

Team composition

CIGen filled the talent and knowledge gap in weeks where the company would have needed months to do so. The line-up included:

Backend Engineers

Frontend Engineer

Project Manager

Solution Architect

The result

Enhancing marketing insights with big data technology

CIGen successfully migrated the client's web application to the cloud and modernized its data processing capabilities.

The project is a classic Big Data solution aimed at simplifying the sorting of millions of data loads daily. The project follows an ETL process scenario for decision support suitable for data warehouses and Business Intelligence tools. Users can upload huge chunks of data loads from various sources and transform them to get some valuable marketing/advertisement insights.

‍

Cloud migration: Seamlessly migrated the application from on-premises to the cloud, ensuring scalability and performance.

Big Data transformation: Transitioned from SQL-based services to Azure Databricks (Spark) for enhanced data processing.

Efficient data handling: Enabled efficient processing and transformation of large datasets for valuable marketing insights.

Integration with Azure Services: Achieved seamless integration with various Azure services for improved data management.

‍

Client about our cooperation

The features

Azure Databricks processing Utilized Azure Databricks for powerful data processing with third-party integrations.

ETL processes Implemented Extract, Transform, and Load (ETL) processes for marketing datasets to generate useful insights.

Versatile output Enabled output of results into various destinations such as Azure Blob, SFTP, and SQL Database based on client needs.

Fast data access Leveraged Azure Blobs for primary data storage, ensuring extremely fast read-write speeds and seamless integration with other Azure services.

‍