my-project-public

repository

loading code, commits, and activity

repositories

loading repo index

#1	---
#2	title: LanceDB
#3	---
#4
#5	## Install Embedchain with LanceDB
#6
#7	Install Embedchain, LanceDB and related dependencies using the following command:
#8
#9	```bash
#10	pip install "embedchain[lancedb]"
#11	```
#12
#13	LanceDB is a developer-friendly, open source database for AI. From hyper scalable vector search and advanced retrieval for RAG, to streaming training data and interactive exploration of large scale AI datasets.
#14	In order to use LanceDB as vector database, not need to set any key for local use.
#15
#16	### With OPENAI
#17	<CodeGroup>
#18
#19	```python main.py
#20	import os
#21	from embedchain import App
#22
#23	# set OPENAI_API_KEY as env variable
#24	os.environ["OPENAI_API_KEY"] = "sk-xxx"
#25
#26	# create Embedchain App and set config
#27	app = App.from_config(config={
#28	"vectordb": {
#29	"provider": "lancedb",
#30	"config": {
#31	"collection_name": "lancedb-index"
#32	}
#33	}
#34	}
#35	)
#36
#37	# add data source and start query in
#38	app.add("https://www.forbes.com/profile/elon-musk")
#39
#40	# query continuously
#41	while(True):
#42	question = input("Enter question: ")
#43	if question in ['q', 'exit', 'quit']:
#44	break
#45	answer = app.query(question)
#46	print(answer)
#47	```
#48
#49	</CodeGroup>
#50
#51	### With Local LLM
#52	<CodeGroup>
#53
#54	```python main.py
#55	from embedchain import Pipeline as App
#56
#57	# config for Embedchain App
#58	config = {
#59	'llm': {
#60	'provider': 'huggingface',
#61	'config': {
#62	'model': 'mistralai/Mistral-7B-v0.1',
#63	'temperature': 0.1,
#64	'max_tokens': 250,
#65	'top_p': 0.1,
#66	'stream': True
#67	}
#68	},
#69	'embedder': {
#70	'provider': 'huggingface',
#71	'config': {
#72	'model': 'sentence-transformers/all-mpnet-base-v2'
#73	}
#74	},
#75	'vectordb': {
#76	'provider': 'lancedb',
#77	'config': {
#78	'collection_name': 'lancedb-index'
#79	}
#80	}
#81	}
#82
#83	app = App.from_config(config=config)
#84
#85	# add data source and start query in
#86	app.add("https://www.tesla.com/ns_videos/2022-tesla-impact-report.pdf")
#87
#88	# query continuously
#89	while(True):
#90	question = input("Enter question: ")
#91	if question in ['q', 'exit', 'quit']:
#92	break
#93	answer = app.query(question)
#94	print(answer)
#95	```
#96
#97	</CodeGroup>
#98
#99
#100	<Snippet file="missing-vector-db-tip.mdx" />

z6Mkq5mY3JWtxoxUobWcfNHm7AkRubgSWEZTkBVqZXJviFZ5/my-project-public