my-project-public

repository

loading code, commits, and activity

repositories

loading repo index

#1	---
#2	title: 🧩 Embedding models
#3	---
#4
#5	## Overview
#6
#7	Embedchain supports several embedding models from the following providers:
#8
#9	<CardGroup cols={4}>
#10	<Card title="OpenAI" href="#openai"></Card>
#11	<Card title="GoogleAI" href="#google-ai"></Card>
#12	<Card title="Azure OpenAI" href="#azure-openai"></Card>
#13	<Card title="AWS Bedrock" href="#aws-bedrock"></Card>
#14	<Card title="GPT4All" href="#gpt4all"></Card>
#15	<Card title="Hugging Face" href="#hugging-face"></Card>
#16	<Card title="Vertex AI" href="#vertex-ai"></Card>
#17	<Card title="NVIDIA AI" href="#nvidia-ai"></Card>
#18	<Card title="Cohere" href="#cohere"></Card>
#19	<Card title="Ollama" href="#ollama"></Card>
#20	<Card title="Clarifai" href="#clarifai"></Card>
#21	</CardGroup>
#22
#23	## OpenAI
#24
#25	To use OpenAI embedding function, you have to set the `OPENAI_API_KEY` environment variable. You can obtain the OpenAI API key from the [OpenAI Platform](https://platform.openai.com/account/api-keys).
#26
#27	Once you have obtained the key, you can use it like this:
#28
#29	<CodeGroup>
#30
#31	```python main.py
#32	import os
#33	from embedchain import App
#34
#35	os.environ['OPENAI_API_KEY'] = 'xxx'
#36
#37	# load embedding model configuration from config.yaml file
#38	app = App.from_config(config_path="config.yaml")
#39
#40	app.add("https://en.wikipedia.org/wiki/OpenAI")
#41	app.query("What is OpenAI?")
#42	```
#43
#44	```yaml config.yaml
#45	embedder:
#46	provider: openai
#47	config:
#48	model: 'text-embedding-3-small'
#49	```
#50
#51	</CodeGroup>
#52
#53	* OpenAI announced two new embedding models: `text-embedding-3-small` and `text-embedding-3-large`. Embedchain supports both these models. Below you can find YAML config for both:
#54
#55	<CodeGroup>
#56
#57	```yaml text-embedding-3-small.yaml
#58	embedder:
#59	provider: openai
#60	config:
#61	model: 'text-embedding-3-small'
#62	```
#63
#64	```yaml text-embedding-3-large.yaml
#65	embedder:
#66	provider: openai
#67	config:
#68	model: 'text-embedding-3-large'
#69	```
#70
#71	</CodeGroup>
#72
#73	## Google AI
#74
#75	To use Google AI embedding function, you have to set the `GOOGLE_API_KEY` environment variable. You can obtain the Google API key from the [Google Maker Suite](https://makersuite.google.com/app/apikey)
#76
#77	<CodeGroup>
#78	```python main.py
#79	import os
#80	from embedchain import App
#81
#82	os.environ["GOOGLE_API_KEY"] = "xxx"
#83
#84	app = App.from_config(config_path="config.yaml")
#85	```
#86
#87	```yaml config.yaml
#88	embedder:
#89	provider: google
#90	config:
#91	model: 'models/embedding-001'
#92	task_type: "retrieval_document"
#93	title: "Embeddings for Embedchain"
#94	```
#95	</CodeGroup>
#96	<br/>
#97	<Note>
#98	For more details regarding the Google AI embedding model, please refer to the [Google AI documentation](https://ai.google.dev/tutorials/python_quickstart#use_embeddings).
#99	</Note>
#100
#101	## AWS Bedrock
#102
#103	To use AWS Bedrock embedding function, you have to set the AWS environment variable.
#104
#105	<CodeGroup>
#106	```python main.py
#107	import os
#108	from embedchain import App
#109
#110	os.environ["AWS_ACCESS_KEY_ID"] = "xxx"
#111	os.environ["AWS_SECRET_ACCESS_KEY"] = "xxx"
#112	os.environ["AWS_REGION"] = "us-west-2"
#113
#114	app = App.from_config(config_path="config.yaml")
#115	```
#116
#117	```yaml config.yaml
#118	embedder:
#119	provider: aws_bedrock
#120	config:
#121	model: 'amazon.titan-embed-text-v2:0'
#122	vector_dimension: 1024
#123	task_type: "retrieval_document"
#124	title: "Embeddings for Embedchain"
#125	```
#126	</CodeGroup>
#127	<br/>
#128	<Note>
#129	For more details regarding the AWS Bedrock embedding model, please refer to the [AWS Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html).
#130	</Note>
#131
#132	## Azure OpenAI
#133
#134	To use Azure OpenAI embedding model, you have to set some of the azure openai related environment variables as given in the code block below:
#135
#136	<CodeGroup>
#137
#138	```python main.py
#139	import os
#140	from embedchain import App
#141
#142	os.environ["OPENAI_API_TYPE"] = "azure"
#143	os.environ["AZURE_OPENAI_ENDPOINT"] = "https://xxx.openai.azure.com/"
#144	os.environ["AZURE_OPENAI_API_KEY"] = "xxx"
#145	os.environ["OPENAI_API_VERSION"] = "xxx"
#146
#147	app = App.from_config(config_path="config.yaml")
#148	```
#149
#150	```yaml config.yaml
#151	llm:
#152	provider: azure_openai
#153	config:
#154	model: gpt-35-turbo
#155	deployment_name: your_llm_deployment_name
#156	temperature: 0.5
#157	max_tokens: 1000
#158	top_p: 1
#159	stream: false
#160
#161	embedder:
#162	provider: azure_openai
#163	config:
#164	model: text-embedding-ada-002
#165	deployment_name: you_embedding_model_deployment_name
#166	```
#167	</CodeGroup>
#168
#169	You can find the list of models and deployment name on the [Azure OpenAI Platform](https://oai.azure.com/portal).
#170
#171	## GPT4ALL
#172
#173	GPT4All supports generating high quality embeddings of arbitrary length documents of text using a CPU optimized contrastively trained Sentence Transformer.
#174
#175	<CodeGroup>
#176
#177	```python main.py
#178	from embedchain import App
#179
#180	# load embedding model configuration from config.yaml file
#181	app = App.from_config(config_path="config.yaml")
#182	```
#183
#184	```yaml config.yaml
#185	llm:
#186	provider: gpt4all
#187	config:
#188	model: 'orca-mini-3b-gguf2-q4_0.gguf'
#189	temperature: 0.5
#190	max_tokens: 1000
#191	top_p: 1
#192	stream: false
#193
#194	embedder:
#195	provider: gpt4all
#196	```
#197
#198	</CodeGroup>
#199
#200	## Hugging Face
#201
#202	Hugging Face supports generating embeddings of arbitrary length documents of text using Sentence Transformer library. Example of how to generate embeddings using hugging face is given below:
#203
#204	<CodeGroup>
#205
#206	```python main.py
#207	from embedchain import App
#208
#209	# load embedding model configuration from config.yaml file
#210	app = App.from_config(config_path="config.yaml")
#211	```
#212
#213	```yaml config.yaml
#214	llm:
#215	provider: huggingface
#216	config:
#217	model: 'google/flan-t5-xxl'
#218	temperature: 0.5
#219	max_tokens: 1000
#220	top_p: 0.5
#221	stream: false
#222
#223	embedder:
#224	provider: huggingface
#225	config:
#226	model: 'sentence-transformers/all-mpnet-base-v2'
#227	model_kwargs:
#228	trust_remote_code: True # Only use if you trust your embedder
#229	```
#230
#231	</CodeGroup>
#232
#233	## Vertex AI
#234
#235	Embedchain supports Google's VertexAI embeddings model through a simple interface. You just have to pass the `model_name` in the config yaml and it would work out of the box.
#236
#237	<CodeGroup>
#238
#239	```python main.py
#240	from embedchain import App
#241
#242	# load embedding model configuration from config.yaml file
#243	app = App.from_config(config_path="config.yaml")
#244	```
#245
#246	```yaml config.yaml
#247	llm:
#248	provider: vertexai
#249	config:
#250	model: 'chat-bison'
#251	temperature: 0.5
#252	top_p: 0.5
#253
#254	embedder:
#255	provider: vertexai
#256	config:
#257	model: 'textembedding-gecko'
#258	```
#259
#260	</CodeGroup>
#261
#262	## NVIDIA AI
#263
#264	[NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) let you quickly use NVIDIA's AI models, such as Mixtral 8x7B, Llama 2 etc, through our API. These models are available in the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), fully optimized and ready to use on NVIDIA's AI platform. They are designed for high speed and easy customization, ensuring smooth performance on any accelerated setup.
#265
#266
#267	### Usage
#268
#269	In order to use embedding models and LLMs from NVIDIA AI, create an account on [NVIDIA NGC Service](https://catalog.ngc.nvidia.com/).
#270
#271	Generate an API key from their dashboard. Set the API key as `NVIDIA_API_KEY` environment variable. Note that the `NVIDIA_API_KEY` will start with `nvapi-`.
#272
#273	Below is an example of how to use LLM model and embedding model from NVIDIA AI:
#274
#275	<CodeGroup>
#276
#277	```python main.py
#278	import os
#279	from embedchain import App
#280
#281	os.environ['NVIDIA_API_KEY'] = 'nvapi-xxxx'
#282
#283	config = {
#284	"app": {
#285	"config": {
#286	"id": "my-app",
#287	},
#288	},
#289	"llm": {
#290	"provider": "nvidia",
#291	"config": {
#292	"model": "nemotron_steerlm_8b",
#293	},
#294	},
#295	"embedder": {
#296	"provider": "nvidia",
#297	"config": {
#298	"model": "nvolveqa_40k",
#299	"vector_dimension": 1024,
#300	},
#301	},
#302	}
#303
#304	app = App.from_config(config=config)
#305
#306	app.add("https://www.forbes.com/profile/elon-musk")
#307	answer = app.query("What is the net worth of Elon Musk today?")
#308	# Answer: The net worth of Elon Musk is subject to fluctuations based on the market value of his holdings in various companies.
#309	# As of March 1, 2024, his net worth is estimated to be approximately $210 billion. However, this figure can change rapidly due to stock market fluctuations and other factors.
#310	# Additionally, his net worth may include other assets such as real estate and art, which are not reflected in his stock portfolio.
#311	```
#312	</CodeGroup>
#313
#314
#315	## Cohere
#316
#317	To use embedding models and LLMs from COHERE, create an account on [COHERE](https://dashboard.cohere.com/welcome/login?redirect_uri=%2Fapi-keys).
#318
#319	Generate an API key from their dashboard. Set the API key as `COHERE_API_KEY` environment variable.
#320
#321	Once you have obtained the key, you can use it like this:
#322
#323	<CodeGroup>
#324
#325	```python main.py
#326	import os
#327	from embedchain import App
#328
#329	os.environ['COHERE_API_KEY'] = 'xxx'
#330
#331	# load embedding model configuration from config.yaml file
#332	app = App.from_config(config_path="config.yaml")
#333	```
#334
#335	```yaml config.yaml
#336	embedder:
#337	provider: cohere
#338	config:
#339	model: 'embed-english-light-v3.0'
#340	```
#341
#342	</CodeGroup>
#343
#344	* Cohere has few embedding models: `embed-english-v3.0`, `embed-multilingual-v3.0`, `embed-multilingual-light-v3.0`, `embed-english-v2.0`, `embed-english-light-v2.0` and `embed-multilingual-v2.0`. Embedchain supports all these models. Below you can find YAML config for all:
#345
#346	<CodeGroup>
#347
#348	```yaml embed-english-v3.0.yaml
#349	embedder:
#350	provider: cohere
#351	config:
#352	model: 'embed-english-v3.0'
#353	vector_dimension: 1024
#354	```
#355
#356	```yaml embed-multilingual-v3.0.yaml
#357	embedder:
#358	provider: cohere
#359	config:
#360	model: 'embed-multilingual-v3.0'
#361	vector_dimension: 1024
#362	```
#363
#364	```yaml embed-multilingual-light-v3.0.yaml
#365	embedder:
#366	provider: cohere
#367	config:
#368	model: 'embed-multilingual-light-v3.0'
#369	vector_dimension: 384
#370	```
#371
#372	```yaml embed-english-v2.0.yaml
#373	embedder:
#374	provider: cohere
#375	config:
#376	model: 'embed-english-v2.0'
#377	vector_dimension: 4096
#378	```
#379
#380	```yaml embed-english-light-v2.0.yaml
#381	embedder:
#382	provider: cohere
#383	config:
#384	model: 'embed-english-light-v2.0'
#385	vector_dimension: 1024
#386	```
#387
#388	```yaml embed-multilingual-v2.0.yaml
#389	embedder:
#390	provider: cohere
#391	config:
#392	model: 'embed-multilingual-v2.0'
#393	vector_dimension: 768
#394	```
#395
#396	</CodeGroup>
#397
#398	## Ollama
#399
#400	Ollama enables the use of embedding models, allowing you to generate high-quality embeddings directly on your local machine. Make sure to install [Ollama](https://ollama.com/download) and keep it running before using the embedding model.
#401
#402	You can find the list of models at [Ollama Embedding Models](https://ollama.com/blog/embedding-models).
#403
#404	Below is an example of how to use embedding model Ollama:
#405
#406	<CodeGroup>
#407
#408	```python main.py
#409	import os
#410	from embedchain import App
#411
#412	# load embedding model configuration from config.yaml file
#413	app = App.from_config(config_path="config.yaml")
#414	```
#415
#416	```yaml config.yaml
#417	embedder:
#418	provider: ollama
#419	config:
#420	model: 'all-minilm:latest'
#421	```
#422
#423	</CodeGroup>
#424
#425	## Clarifai
#426
#427	Install related dependencies using the following command:
#428
#429	```bash
#430	pip install --upgrade 'embedchain[clarifai]'
#431	```
#432
#433	set the `CLARIFAI_PAT` as environment variable which you can find in the [security page](https://clarifai.com/settings/security). Optionally you can also pass the PAT key as parameters in LLM/Embedder class.
#434
#435	Now you are all set with exploring Embedchain.
#436
#437	<CodeGroup>
#438
#439	```python main.py
#440	import os
#441	from embedchain import App
#442
#443	os.environ["CLARIFAI_PAT"] = "XXX"
#444
#445	# load llm and embedder configuration from config.yaml file
#446	app = App.from_config(config_path="config.yaml")
#447
#448	#Now let's add some data.
#449	app.add("https://www.forbes.com/profile/elon-musk")
#450
#451	#Query the app
#452	response = app.query("what college degrees does elon musk have?")
#453	```
#454	Head to [Clarifai Platform](https://clarifai.com/explore/models?page=1&perPage=24&filterData=%5B%7B%22field%22%3A%22output_fields%22%2C%22value%22%3A%5B%22embeddings%22%5D%7D%5D) to explore all the State of the Art embedding models available to use.
#455	For passing LLM model inference parameters use `model_kwargs` argument in the config file. Also you can use `api_key` argument to pass `CLARIFAI_PAT` in the config.
#456
#457	```yaml config.yaml
#458	llm:
#459	provider: clarifai
#460	config:
#461	model: "https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct"
#462	model_kwargs:
#463	temperature: 0.5
#464	max_tokens: 1000
#465	embedder:
#466	provider: clarifai
#467	config:
#468	model: "https://clarifai.com/clarifai/main/models/BAAI-bge-base-en-v15"
#469	```
#470	</CodeGroup>

z6Mkq5mY3JWtxoxUobWcfNHm7AkRubgSWEZTkBVqZXJviFZ5/my-project-public