Food & Beverage Company
Harnessing Real-Time Data with Databricks Reduces Waste and Enhances Forecasting
Syniti’s seamless integration of Databricks with our existing ERP systems has provided us with a reliable, single source of truth for our data. We’ve seen a remarkable improvement in our ability to monitor and analyze in real-time have made it easier for our teams to access up-to-date dashboards and insights, resulting in reduced waste, enhanced data governance, and improved operational agility.
Company Background
This food & beverage company is a pork producer and food processor based in Smithfield, Virginia. It is the world’s largest producer of pork and pigs. As the largest pork processor in the world, they are uniquely positioned to deliver differentiated products to meet customer specifications—both domestically and abroad. Vertical integration is a key point of difference for their products and brands, allowing the company to drive changes through the supply chain.
Project Background
The food industry is uniquely challenged with a strict timeframe. With only a six-week life cycle between the time a food product is packaged, shipped, stored, and sold, delays in reporting and data easily risk spoilage, making items no longer viable to sell. Ensuring the team was referencing the most up-to-date data to monitor where items are in the production process is vital in avoiding unnecessary waste. But given constraints like narrow shipping windows and differences in item perishability and the variable (but generally rising) cost of fuel and labor, easy solutions remain elusive.
The food & beverage company needed the tools to move data across in a real-time fashion for critical reporting, forecasting, and analytics. Originally using SQL Server as the backbone for their data, the new connection to Databricks would provide them with the data engineering tools needed for processing and transforming their vast volumes of data to build machine learning models.
Partnering with Syniti has been a game-changer for our operations. With the implementation of Databricks Lakehouse Platform, we’ve transformed our data processes, allowing us to access realtime insights that were previously out of reach. The ability to merge IoT data with ERP systems has streamlined our forecasting and reporting, giving our teams the tools they need to make informed decisions quickly. The impact on our production efficiency and waste reduction has been significant, and we are now better equipped to meet the demands of a fast-paced food industry.
Implementation Highlights
Syniti’s data management consultants leveraged the Databricks Lakehouse Platform to help solve for their critical business challenges, build forecasting solutions, and help maneuver the challenges of ingesting and converging operational technology (OT) data with ERP data.
The project also involved converting and replicating data to -be consumed by the company’s planning and forecasting implementation. Syniti implemented data replication solutions to replicate data from relational database tables to Databricks on Azure to perform one-time, continuous, or incremental replication. Low-code change data capture (CDC) supports high-speed replication of data from on-prem legacy sources, databases, -and mainframes and low-code data ingestion from data platforms.
As a result, data is mirrored from the ERP systems and into a more consumable, up-to-date, real-time dashboard so the team always has access to the most fresh, accurate data. The process supports Smithfield’s active data governance on materials, customers and vendors, as data is pushed to the data lake every 5 minutes for real-time reporting and business analysis.
Results
Through Syniti’s own suite of data migration and integration solutions, Syniti enabled deeper insights, wider adoption, and improved results.
By leveraging the Databricks Lakehouse, Syniti was able to accelerate the connection from high-velocity IoT data to external systems like ERPs and enabled the company to converge their data infrastructure for more advanced analytics while providing a single source of truth. Databricks centralized platform supports data governance and processing, enabling data sharing across the organization and collaboration on data projects. Users now have the ability to work with a variety of custom dashboards, resulting in improved data insights and reporting. The resource also helps to inform error and audit reporting on materials such as food products or animal stock.