OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Data Ingestion for the RAG from Dynamo DB to AuroraDB with pgVector to store embeddings

  • Thread starter Thread starter Yash Kamal Saxena
  • Start date Start date
Y

Yash Kamal Saxena

Guest
I have data stored in my DynamoDB which is frequently updated through back-end services. Now I want to create a PG vector based AuroraDB vector database for storing embeddings, which I want to be automatically updated whenever there is the change in the DynamoDB.

I thought about using the EventBridge but need more suggestion on that.

My aim is to create the new embeddings everytime there is the change (Upsert) in the DynamoDB and store them in the PG Vector Database. So that I can perform the RAG on latest embeddings to so the answer from LLM must be context aware.

In the phase of architectural designing and ideation of this feature.
<p>I have data stored in my <strong>DynamoDB</strong> which is frequently updated through back-end services. Now I want to create a PG vector based <strong>AuroraDB vector database</strong> for storing <strong>embeddings</strong>, which I want to be automatically updated whenever there is the change in the DynamoDB.</p>
<p>I thought about using the <strong>EventBridge</strong> but need more suggestion on that.</p>
<p>My aim is to create the new embeddings everytime there is the change (Upsert) in the DynamoDB and store them in the PG Vector Database. So that I can perform the RAG on <strong>latest embeddings</strong> to so the answer from LLM must be context aware.</p>
<p>In the phase of architectural designing and ideation of this feature.</p>
Continue reading...
 

Latest posts

Online statistics

Members online
0
Guests online
4
Total visitors
4
Top