Centralize your data with Squid integrations

In many large-scale legacy systems that exist today, projects use tens (if not hundreds) of data sources in their tech stack. Related data can be spread across entirely different data providers, causing pain points in data access and analysis processes. Engineering and IT teams have to devote precious time and resources to maintaining these databases in order to ensure that their data is reliable and protected. But wouldn’t it be great if you could have all your data in one place?

One of the key benefits of Squid’s platform is the ability to connect to multiple data sources and centralize your data. This effectively mitigates the challenges associated with managing multiple data sources.

Here at Squid we know that each developer’s needs vary greatly. Squid’s decision to create integrations that best serve each customer’s unique needs is what makes Squid the most impactful middle-tier solution available. A Squid integration allows you to connect your own databases, API endpoints, authentication providers, and more to the Squid platform in order to make them accessible in your client application. To see a full list of available integrations, check out the Available integrations tab of the console.

In an effort to show off the power of Squid integrations, I created a simple Squid + React app that connects two external data sources in order to answer a simple (yet crucial) nautical question: where in the world can you travel in order to see both the magnificent tentacled underwater creature AND a decades old shipwreck?

Fortunately, we have plenty of data to help us answer this question. MongoDB sample data includes a shipwrecks collection, while the Squid Squad has stored scientific squid sightings in a Snowflake database. But how do we deal with these two different data sources?

The first step in centralizing the data is to add the database integrations in the console. We will have two integrations – one for MongoDB and one for Snowflake. Setting up an integration in the console is easy and only requires a few configuration options for connection to the external database. For specific instructions on how to connect an integration, please view our documentation.

Once we finalize the integration, we can now access our data collections directly from the Squid SDKs! Since we are building a React client, we will use the Squid React SDK. The Squid React SDK contains hooks for accessing and querying our data. The following query will simply return all the data in the MongoDB shipwrecks collection:

const mongoCollection = useCollection<ShipwreckType>("shipwrecks", "mongo"); 
// "shipwrecks" is the collection name, "mongo" is the integration ID

const { data } = useQuery(mongoCollection.query());

Using the coordinates of a random shipwreck from MongoDB, we can then query the Snowflake database and see if there are any recorded squid sightings within 1º of square area. Sorting by the year will show us the most recent squid sightings first:

const { data } = useQuery(
  snowflakeCollection
    .query()
    .gt("DECIMALLATITUDE", coordinates[0] - 1)
    .lt("DECIMALLATITUDE", coordinates[0] + 1)
    .gt("DECIMALLONGITUDE", coordinates[1] - 1)
    .lt("DECIMALLONGITUDE", coordinates[1] + 1)
    .sortBy("YEAR", false)
);

Here is a live demo of the project:

We have now successfully integrated our two databases and are able to search for recorded squid sightings close to underwater shipwrecks! One advantage of Squid integrations is that they allow for incremental adoption; while we integrated these two databases, if we have other databases in the project, those do not need to be integrated at the same time. Instead, we can make these changes gradually, modernizing our tech stack one integration at a time.

Bonus section

Query with Squid AI is an extremely powerful tool where you can ask Squid AI questions about your data in human-readable sentences. Using Squid AI, we can query our integrated databases and receive both the results and the step-by-step process Squid AI used to run the query. By using Squid, you too can harness the power of Squid AI and gain valuable insights on your data.

And that’s it! Thanks for reading. If you’re looking to learn more or join our community check out Squid’s documentation or Discord channel.