Object Store in Mulesoft

Dealing with duplicate data entries and inefficient workflows can be more taxing than the task itself. Don’t you agree? These setbacks highlight the importance of streamlining your data management process and ensuring accurate data handling. If you find yourself nodding along and are curious to find some valuable insights, you are in the right place! This blog post will help you discover the full potential of Object Store in MuleSoft - the solution you have been searching for. We will explore the different types of data storage, how it works, and the various operations it can perform. But that’s not all. We will also take you through a real-world use case where Object Store helped a healthcare company avoid duplicate file integrations and manage data efficiently. Excited? Let’s get started!

What is Object Store?

In MuleSoft, an Object Store is a storage mechanism used for storing and retrieving data within a Mule application. It provides a way to persistently store data across different executions of the application. Object Stores in Mule are typically used to store temporary data, session state, or any other kind of data that needs to be shared across different parts of the application. This is the default Object Store Connector in MuleSoft, and you can also create custom Object Stores as needed.

Types of Data Storage in Object Store

When working with MuleSoft’s Object Store, understanding the types of data storage available is significant for effectively managing your data. Object Store in Mule offers two main types of storage – persistent and non-persistent – each with its own characteristics and use cases. Let’s explore these in detail:

Persistent – In persistent Object Store, the data is stored on disk, either locally or in a specified location like a database or file system. This ensures that the data persists even if the application is restarted or if there are failures.

Non-Persistent – In a non-persistent Object Store, data is stored temporarily in memory, without using external storage like a database or file system. This data will be lost in case of an application restart or Mule runtime crashes. Data sharing is not allowed if you choose a transient Object Store.

How Object Store works?

An Object Store serves as a simple key-value store where data can be stored and retrieved based on a unique identifier (key). Object Stores are commonly used to store temporary data, such as state information, session data, cached results or timestamps across different components or flows within a Mule application.

What Operations Can Object Store Perform?

MuleSoft’s Object Store is a versatile tool that supports a variety of operations for managing your data efficiently. Whether you need to store, retrieve, or manage your data, Object Store in Mule 4 provides a set of operations to help you handle these tasks seamlessly.

Let’s explore the different operations you can perform with Object Store:

1. Store: Stores the given value using the given key.
2. Retrieve: Retrieves the value stored for the given key. If there is no key with the given name, the default value is returned.
3. Retrieve All Keys: Returns a list of all keys that the object store currently holds values for.
4. Retrieve All: Retrieves all the key-value pairs present in the Object Store.
5. Contains: Checks if there is any value associated with the given key. If no value exists for the key, false is returned.
6. Remove: Removes the value associated with the given key from the Object Store.
7. Clear: Clears all the contents in the store, removing all keys and values.

Read More: Introduction to MuleSoft's Code Builder

Configuration of Object Store Connector

Configuring the Object Store connector in MuleSoft is essential to leverage its full capabilities. The Object Store connector offers a range of features that facilitate efficient data storage and retrieval within your MuleSoft application. Here are some of the key features provided by Object Store connector as seen in the table below.

Limitations of Object Store

It is important to note that while Object Store is a powerful tool for managing temporary data, it may not be suitable for every use case, especially when dealing with large volumes of data.

Object Store provides limited storage capacity, primarily designed for temporary data storage. Therefore, it is essential to consider the scalability and specific requirements of your application when deciding to use Object Store for data storage.

Exploring Object Store Through a Use Case in Healthcare Integration

Let’s understand the functioning of Object Store with the help of the following use case:

Problem Statement: A Healthcare Company's Challenge with Patient Details Integration

The healthcare company faces difficulties integrating patient details files into the heath cloud from the SFTP server. The absence of a standardized naming convention to distinguish between new and old files poses a significant challenge. Repeated integration of the same file could lead to duplicate entries in the health cloud, causing confusion and contradicting best practices.

Solution: Using Object Store

To avoid integrating duplicate files into the health cloud (Salesforce), we leverage Object Store as our solution. Here's how it works:

Every day, we store the timestamp in the Object Store representing the last time we handled a file. When the API runs the next day, it first retrieves this stored timestamp from the Object Store. Then, it compares the timestamp with the creation or modification time of the files. Only files created or updated after the recorded time are processed. This system ensures that we don’t integrate the sale file multiple times, thus preventing duplicate entries in the health cloud.

How We implemented Object Store to Avoid File Duplicity

In our application, we've implemented Object Store to prevent file duplicity effectively. Here's how it works:

When the API runs for the first time, it stores the current time value (default value) in the form of a key-value pair in Object Store. For example, if our API runs for the first time on 01-01-2024 at 9:00 AM IST, the timestamp "01-01-2024 09:00:00" will be stored in Object Store as the key-value pair "timestamp": "01-01-2024 09:00:00". This initial timestamp serves as a reference point for subsequent runs of the API. And it will integrate all the files from SFTP to Health Cloud (Salesforce).

Upon the API's second run, we retrieve the timestamp value stored in Object Store from the previous day. Next, we compare this retrieved timestamp with the last updated or created value of the files coming from the SFTP server. The logic is as follows: If the files have a last update or create value greater than the timestamp value stored in Object Store, only those files will be integrated into the health cloud (Salesforce). Files that have already been integrated into the health cloud will be discarded, preventing duplicity and ensuring efficient data management.

[Note: We have kept the max entry as 1 because we only need one value of timestamp which is latest and it will keep on replacing the last value of timestamp with the latest value, so at the end of the flow there will be only one value in the object store].

Steps taken to implement the solution effectively:

• Before fetching files from SFTP, we retrieve the last scheduled run timestamp from Object Store. If our application is running for the very first time, we can use a default timestamp as the initial value.
• The retrieved timestamp is stored in a variable named ‘timestamp’. This variable is then used to filter the files coming from SFTP, ensuring that only files with a last update or create value greater than the stores timestamp are processed.
• After processing the files, the current scheduled run timestamp is again stored in Object Store. This updated timestamp becomes the reference point for the next run to retrieve files, effectively eliminating file redundancy and ensuring only new or updated files are integrated into the health cloud (Salesforce).

Results:

The implemented solution, featuring an Object Store connector storing the timestamp of the last scheduled run and subsequent file filtering based on this timestamp, delivers a multitude of benefits:

• Elimination of Duplicate Data Entries: Ensuring that only new or updated files are integrated, thereby avoiding redundant data entries.
• Optimization of Operational Efficiency: Automation of processes has streamlined operations, reducing manual efforts and increasing productivity.
• Enhancement of Data Integrity: Preventing redundant transfers has improved data accuracy and consistency.
• Streamlining Workflow: Seamless integration between the SFTP server and Salesforce has improved overall workflow efficiency.
• Facilitation of Compliance Adherence: Accurate data handling practices have ensured compliance with regulatory standards.
• Improved User Satisfaction: Reduced errors and smoother data integration have enhanced user experience and satisfaction.
• Mitigation of Operational Risks: Proactive measures have minimized risks associated with data redundancy and errors.
• Solving Storage Space Issues: Efficient handling of data has optimized storage space usage, reducing the risk of storage constraints.

Collectively, these outcomes have ensured the reliability, performance, and effectiveness of the application, leading to improved user experiences.

Transform Your Data Management Strategies with Object Store Today!

As we conclude this exploration of Object Store and its impact on data integration, we invite you to envision the possibilities it holds for your organization, regardless of your industry. Object Store offers a pathway to enhanced reliability, improved performance, and a seamless user experience. Whether you are grappling with data duplication, workflow inefficiencies, or compliance requirements, Object Store provides a solution that can drive tangible improvements in your processes.

Take the first step towards exploring the potential of Object Store and elevate your data management strategies. Get started today and experience the difference firsthand with a dedicated MuleSoft partner like Accelirate. Let's empower your organization and teams together!