Use DeepSeek with Amazon OpenSearch Service vector database and Amazon SageMaker

DeepSeek-R1 is a robust and cost-effective AI mannequin that excels at advanced reasoning duties. When mixed with Amazon OpenSearch Service, it allows strong Retrieval Augmented Era (RAG) purposes. This submit reveals you arrange RAG utilizing DeepSeek-R1 on Amazon SageMaker with an OpenSearch Service vector database because the information base. This instance gives an answer for enterprises seeking to improve their AI capabilities.

OpenSearch Service gives wealthy capabilities for RAG use circumstances, in addition to vector embedding-powered semantic search. You should utilize the versatile connector framework and search circulate pipelines in OpenSearch to hook up with fashions hosted by DeepSeek, Cohere, and OpenAI, in addition to fashions hosted on Amazon Bedrock and SageMaker. On this submit, we construct a connection to DeepSeek’s textual content era mannequin, supporting a RAG workflow to generate textual content responses to consumer queries.

Resolution overview

The next diagram illustrates the answer structure.

On this walkthrough, you’ll use a set of scripts to create the previous structure and information circulate. First, you’ll create an OpenSearch Service area, and deploy DeepSeek-R1 to SageMaker. You’ll execute scripts to create an AWS Identification and Entry Administration (IAM) position for invoking SageMaker, and a task to your consumer to create a connector to SageMaker. You’ll create an OpenSearch connector and mannequin that may allow the retrieval_augmented_generation processor inside OpenSearch to execute a consumer question, carry out a search, and use DeepSeek to generate a textual content response. You’ll create a connector to SageMaker with Amazon Titan Textual content Embeddings V2 to create embeddings for a set of paperwork with inhabitants statistics. Lastly, you’ll execute the question to match inhabitants development in Miami and New York Metropolis.

Conditions

We’ve created and open-sourced a GitHub repo with all of the code it’s essential comply with together with the submit and deploy it for your self. You have to the next stipulations:

Deploy DeepSeek on Amazon SageMaker

You have to to have or deploy DeepSeek with an Amazon SageMaker endpoint. To be taught extra about deploying DeepSeek-R1 on SageMaker, consult with Deploying DeepSeek-R1 Distill Mannequin on AWS utilizing Amazon SageMaker AI.

Create an OpenSearch Service area

Discuss with Create an Amazon OpenSearch Service area for directions on create your area. Make word of the area Amazon Useful resource Title (ARN) and area endpoint, each of which could be discovered within the Basic data part of every area on the OpenSearch Service console.

Obtain and put together the code

Run the next steps out of your native laptop or workspace that has Python and git:

1. In the event you haven’t already, clone the repo into a neighborhood folder utilizing the next command:

git clone https://github.com/Jon-AtAWS/opensearch-examples.git

2. Create a Python digital surroundings:

cd opensearch-examples/opensearch-deepseek-rag
python -m venv .venv
supply .venv/bin/activate
pip set up -r necessities.txt

The instance scripts use surroundings variables for setting some widespread parameters. Set these up now utilizing the next instructions. Remember to replace along with your AWS Area, your SageMaker endpoint ARN and URL, your OpenSearch Service area’s endpoint and ARN, and your area’s main consumer and password.

export DEEPSEEK_AWS_REGION=''
export SAGEMAKER_MODEL_INFERENCE_ARN='' 
export SAGEMAKER_MODEL_INFERENCE_ENDPOINT=''
export OPENSEARCH_SERVICE_DOMAIN_ARN='’
export OPENSEARCH_SERVICE_DOMAIN_ENDPOINT=''
export OPENSEARCH_SERVICE_ADMIN_USER=''
export OPENSEARCH_SERVICE_ADMIN_PASSWORD=''

You now have the code base and have your digital surroundings arrange. You possibly can study the contents of the opensearch-deepseek-rag listing. For readability of function and studying, we’ve encapsulated every of seven steps in its personal Python script. This submit will information you thru working these scripts. We’ve additionally chosen to make use of surroundings variables to go parameters between scripts. In an precise answer, you’d encapsulate the code in lessons and go the values the place wanted. Coding this fashion is clearer, however is much less environment friendly and doesn’t comply with coding finest practices. Use these scripts as examples to tug from.

First, you’ll arrange permissions to your OpenSearch Service area to hook up with your SageMaker endpoint.

Arrange permissions

You’ll create two IAM roles. The primary will enable OpenSearch to name your SageMaker endpoint. The second will can help you make the create connector API name to OpenSearch.

1. Look at the code in create_invoke_role.py.
2. Return to the command line, and execute the script:

python create_invoke_role.py

3. Execute the command line from the script’s output to set the INVOKE_DEEPSEEK_ROLE surroundings variable.

You’ve created a task named invoke_deepseek_role, with a belief relationship for OpenSearch Service to imagine the position, and with a permission coverage that enables OpenSearch Service to invoke your SageMaker endpoint. The script outputs the ARNs to your position and coverage and moreover a command line command so as to add the position to your surroundings. Execute that command earlier than working the subsequent script. Make an observation of the position ARN in case it’s essential return at a later time.

Now it’s essential create a task to your consumer to have the ability to create a connector in OpenSearch Service.

4. Look at the code in create_connector_role.py.
5. Return to the command line and execute the script:

python create_connector_role.py

6. Execute the command line from the script’s output to set the CREATE_DEEPSEEK_CONNECTOR_ROLE surroundings variable.

You’ve created a task named create_deepseek_connector_role, with a belief relationship with the present consumer and permissions to put in writing to OpenSearch Service. You want these permissions to name the OpenSearch create_connector API, which packages a connection to a distant mannequin host, DeepSeek on this case. The script prints the coverage’s and position’s ARNs, and moreover a command line command so as to add the position to your surroundings. Execute that command earlier than working the subsequent script. Once more, make word of the position ARN, simply in case.

Now that you’ve your roles created, you’ll inform OpenSearch about them. The fine-grained entry management function contains an OpenSearch position, ml_full_access, that may enable authenticated entities to execute API calls inside OpenSearch.

7. Look at the code in setup_opensearch_security.py.
8. Return to the command line and execute the script:

python setup_opensearch_security.py

You arrange the OpenSearch Service safety plugin to acknowledge two AWS roles: invoke_create_connector_role and LambdaInvokeOpenSearchMLCommonsRole. You’ll use the second position later, if you join with an embedding mannequin and cargo information into OpenSearch to make use of as a RAG information base. Now that you’ve permissions in place, you possibly can create the connector.

Create the connector

You create a connector with configuration that tells OpenSearch join, gives credentials for the goal mannequin host, and gives immediate particulars. For extra data, see Creating connectors for third-party ML platforms.

1. Look at the code in create_connector.py.
2. Return to the command line and execute the script:

python create_connector.py

3. Execute the command line from the script’s output to set the DEEPSEEK_CONNECTOR_ID surroundings variable.

The script will create the connector to name the SageMaker endpoint and return the connector ID. The connector is an OpenSearch assemble that tells OpenSearch how to hook up with an exterior mannequin host. You don’t use it immediately; you create an OpenSearch mannequin for that.

Create an OpenSearch mannequin

If you work with machine studying (ML) fashions, in OpenSearch, you employ OpenSearch’s ml-commons plugin to create a mannequin. ML fashions are an OpenSearch abstraction that allow you to carry out ML duties like sending textual content for embeddings throughout indexing, or calling out to a big language mannequin (LLM) to generate textual content in a search pipeline. The mannequin interface gives you with a mannequin ID in a mannequin group that you simply then use in your ingest pipelines and search pipelines.

1. Look at the code in create_deepseek_model.py.
2. Return to the command line and execute the script:

python create_deepseek_model.py

3. Execute the command line from the script’s output to set the DEEPSEEK_MODEL_ID surroundings variable.

You created an OpenSearch ML mannequin group and mannequin that you should utilize to create ingest and search pipelines. The _register API locations the mannequin within the mannequin group and references your SageMaker endpoint by means of the connector (connector_id) you created.

Confirm your setup

You possibly can run a question to confirm your setup and just be sure you can connect with DeepSeek on SageMaker and obtain generated textual content. Full the next steps:

1. On the OpenSearch Service console, select Dashboard underneath Managed clusters within the navigation pane.
2. Select your area’s dashboard.

3. Select the OpenSearch Dashboards URL (twin stack) hyperlink to open OpenSearch Dashboards.
4. Log in to OpenSearch Dashboards along with your main consumer identify and password.
5. Dismiss the welcome dialog by selecting Discover by myself.
6. Dismiss the brand new feel and look dialog.
7. Verify the worldwide tenant within the Choose your tenant dialog.
8. Navigate to the Dev Instruments tab.
9. Dismiss the welcome dialog.

You too can get to Dev Instruments by increasing the navigation menu (three traces) to disclose the navigation pane, and scrolling all the way down to Dev Instruments.

The Dev Instruments web page gives a left pane the place you enter REST API calls. You execute the instructions and the best pane reveals the output of the command. Enter the next command within the left pane, exchange your_model_id with the mannequin ID you created, and run the command by inserting the cursor anyplace within the command and selecting the run icon.

POST _plugins/_ml/fashions//_predict{  "parameters": {    "inputs": "Hi there"  }}

It’s best to see output like the next screenshot.

Congratulations! You’ve now created and deployed an ML mannequin that may use the connector you created to name to your SageMaker endpoint, and use DeepSeek to generate textual content. Subsequent, you’ll use your mannequin in an OpenSearch search pipeline to automate a RAG workflow.

Arrange a RAG workflow

RAG is a manner of including data to the immediate in order that the LLM producing the response is extra correct. An total generative utility like a chatbot orchestrates a name to exterior information bases and augments the immediate with information from these sources. We’ve created a small information base comprising inhabitants data.

OpenSearch gives search pipelines, that are units of OpenSearch search processors which might be utilized to the search request sequentially to construct a ultimate consequence. OpenSearch has processors for hybrid search, reranking, and RAG, amongst others. You outline your processor after which ship your queries to the pipeline. OpenSearch responds with the ultimate consequence.

If you construct a RAG utility, you select a information base and a retrieval mechanism. Typically, you’ll use an OpenSearch Service vector database as a information base, performing a k-nearest neighbor (k-NN) search to include semantic data within the retrieval with vector embeddings. OpenSearch Service gives integrations with vector embedding fashions hosted in Amazon Bedrock and SageMaker (amongst different choices).

Make it possible for your area is working OpenSearch 2.9 or later, and that fine-grained entry management is enabled for the area. Then full the next steps:

1. On the OpenSearch Service console, select Integrations within the navigation pane.
2. Select Configure area underneath Integration with textual content embedding fashions by means of Amazon SageMaker.

3. Select Configure public area.
4. In the event you created a digital personal cloud (VPC) area as a substitute, select Configure VPC area.

You can be redirected to the AWS CloudFormation console.

5. For Amazon OpenSearch Endpoint, enter your endpoint.
6. Go away every part else as default values.

The CloudFormation stack requires a task to create a connector to the all-MiniLM-L6-v2 mannequin, hosted on SageMaker, referred to as LambdaInvokeOpenSearchMLCommonsRole. You enabled entry for this position if you ran setup_opensearch_security.py. In the event you modified the identify in that script, you’ll want to change it within the Lambda Invoke OpenSearch ML Commons Function Title area.

7. Choose I acknowledge that AWS CloudFormation may create IAM sources with customized names, and select Create stack.

For simplicity, we’ve elected to make use of the open supply all-MiniLM-L6-v2 mannequin, hosted on SageMaker for embedding era. To realize excessive search high quality for manufacturing workloads, you need to fine-tune light-weight fashions like all-MiniLM-L6-v2, or use OpenSearch Service integrations with fashions reminiscent of Cohere Embed V3 on Amazon Bedrock or Amazon Titan Textual content Embedding V2, that are designed to ship excessive out-of-the-box high quality.

Look ahead to CloudFormation to deploy your stack and the standing to vary to Create_Complete.

8. Select the stack’s Outputs tab on the CloudFormation console and replica the worth for ModelID.

You’ll use this mannequin ID to attach along with your embedding mannequin.

9. Look at the code in load_data.py.
10. Return to the command line and set an surroundings variable with the mannequin ID of the embedding mannequin:

export EMBEDDING_MODEL_ID=''

11. Execute the script to load information into your area:

The script creates the population_data index and an OpenSearch ingest pipeline that calls SageMaker utilizing the connector referenced by the embedding mannequin ID. The ingest pipeline’s area mapping tells OpenSearch the supply and vacation spot fields for every doc’s embedding.

Now that you’ve your information base ready, you possibly can run a RAG question.

12. Look at the code in run_rag.py.
13. Return to the command line and execute the script:

The script creates a search pipeline with an OpenSearch retrieval_augmented_generation processor. The processor automates working an OpenSearch k-NN question to retrieve related data and including that data to the immediate. It makes use of the generation_model_id and connector to the DeepSeek mannequin on SageMaker to generate a textual content response for the consumer’s query. The OpenSearch neural question (line 55 of run_rag.py) takes care of producing the embedding for the k-NN question utilizing the embedding_model_id. Within the ext part of the question, you present the consumer’s query for the LLM. The llm_model is ready to bedrock/claude as a result of the parameterization and actions are the identical as they’re for DeepSeek. You’re nonetheless utilizing DeepSeek to generate textual content.

Look at the output from OpenSearch Service. The consumer requested the query “What’s the inhabitants improve of New York Metropolis from 2021 to 2023? How is the trending evaluating with Miami?” The primary portion of the consequence reveals the hits—paperwork OpenSearch retrieved from the semantic question—because the inhabitants statistics for New York Metropolis and Miami. The subsequent part of the response contains the immediate, in addition to DeepSeek’s reply.

Okay, so I want to determine the inhabitants improve of New York Metropolis from 2021 to 2023 and examine it with Miami's development. 
Let me begin by wanting on the information offered within the search outcomes.
From SEARCH RESULT 2, I see that in 2021, NYC had a inhabitants of 18,823,000. 
In 2022, it was 18,867,000, and in 2023, it is 18,937,000. 
So, the rise from 2021 to 2022 is eighteen,867,000 - 18,823,000 = 44,000. 
Then from 2022 to 2023, it is 18,937,000 - 18,867,000 = 70,000. 
Including these collectively, the entire improve from 2021 to 2023 is 44,000 + 70,000 = 114,000.
Now, taking a look at Miami's information in SEARCH RESULT 1. In 2021, Miami's inhabitants was 6,167,000, in 2022 it was 6,215,000, and in 2023 it is 6,265,000. 
The rise from 2021 to 2022 is 6,215,000 - 6,167,000 = 48,000. From 2022 to 2023, it is 6,265,000 - 6,215,000 = 50,000. 
So, the entire improve is 48,000 + 50,000 = 98,000.Evaluating the 2, NYC's improve of 114,000 is larger than Miami's 98,000. 
So, NYC's inhabitants elevated extra over that interval."

Congratulations! You’ve linked to an embedding mannequin, created a information base, and used that information base, together with DeepSeek, to generate a textual content response to a query on inhabitants adjustments in New York Metropolis and Miami. You possibly can adapt the code from this submit to create your individual information base and run your individual queries.

Clear up

To keep away from incurring extra expenses, clear up the sources you deployed:

1. Delete the SageMaker deployment of DeepSeek. For directions, see Cleansing Up.
2. In case your Jupyter pocket book has misplaced context, you possibly can delete the endpoint:
   1. On the SageMaker console, underneath Inference within the navigation pane, select Endpoints.
   2. Choose your endpoint and select Delete.
3. Delete the CloudFormation template for connecting to SageMaker for the embedding mannequin.
4. Delete the OpenSearch Service area you created.

Conclusion

The OpenSearch connector framework is a versatile manner so that you can entry fashions you host on different platforms. On this instance, you linked to the open supply DeepSeek mannequin that you simply deployed on SageMaker. DeepSeek’s reasoning capabilities, augmented with a information base within the OpenSearch Service vector engine, enabled it to reply a query evaluating inhabitants development in New York and Miami.

Discover out extra about AI/ML capabilities of OpenSearch Service, and tell us how you’re utilizing DeepSeek and different generative fashions to construct!

In regards to the Authors

Jon Handler is the Director of Options Structure for Search Companies at Amazon Net Companies, based mostly in Palo Alto, CA. Jon works carefully with OpenSearch and Amazon OpenSearch Service, offering assist and steering to a broad vary of shoppers who’ve search and log analytics workloads for OpenSearch. Previous to becoming a member of AWS, Jon’s profession as a software program developer included 4 years of coding a large-scale, eCommerce search engine. Jon holds a Bachelor of the Arts from the College of Pennsylvania, and a Grasp of Science and a Ph. D. in Pc Science and Synthetic Intelligence from Northwestern College.

Yaliang Wu is a Software program Engineering Supervisor at AWS, specializing in OpenSearch initiatives, machine studying, and generative AI purposes.