my-project-public

repository

loading code, commits, and activity

repositories

loading repo index

#1	---
#2	title: Hugging Face Reranker
#3	description: 'Access thousands of reranking models from Hugging Face Hub'
#4	---
#5
#6	## Overview
#7
#8	The Hugging Face reranker provider gives you access to thousands of reranking models available on the Hugging Face Hub. This includes popular models like BAAI's BGE rerankers and other state-of-the-art cross-encoder models.
#9
#10	## Configuration
#11
#12	### Basic Setup
#13
#14	```python
#15	from mem0 import Memory
#16
#17	config = {
#18	"reranker": {
#19	"provider": "huggingface",
#20	"config": {
#21	"model": "BAAI/bge-reranker-base",
#22	"device": "cpu"
#23	}
#24	}
#25	}
#26
#27	m = Memory.from_config(config)
#28	```
#29
#30	### Configuration Parameters
#31
#32	\| Parameter \| Type \| Default \| Description \|
#33	\|-----------\|------\|---------\|-------------\|
#34	\| `model` \| str \| Required \| Hugging Face model identifier \|
#35	\| `device` \| str \| "cpu" \| Device to run model on ("cpu", "cuda", "mps") \|
#36	\| `batch_size` \| int \| 32 \| Batch size for processing \|
#37	\| `max_length` \| int \| 512 \| Maximum input sequence length \|
#38	\| `trust_remote_code` \| bool \| False \| Allow remote code execution \|
#39
#40	### Advanced Configuration
#41
#42	```python
#43	config = {
#44	"reranker": {
#45	"provider": "huggingface",
#46	"config": {
#47	"model": "BAAI/bge-reranker-large",
#48	"device": "cuda",
#49	"batch_size": 16,
#50	"max_length": 512,
#51	"trust_remote_code": False,
#52	"model_kwargs": {
#53	"torch_dtype": "float16"
#54	}
#55	}
#56	}
#57	}
#58	```
#59
#60	## Popular Models
#61
#62	### BGE Rerankers (Recommended)
#63
#64	```python
#65	# Base model - good balance of speed and quality
#66	config = {
#67	"reranker": {
#68	"provider": "huggingface",
#69	"config": {
#70	"model": "BAAI/bge-reranker-base",
#71	"device": "cuda"
#72	}
#73	}
#74	}
#75
#76	# Large model - better quality, slower
#77	config = {
#78	"reranker": {
#79	"provider": "huggingface",
#80	"config": {
#81	"model": "BAAI/bge-reranker-large",
#82	"device": "cuda"
#83	}
#84	}
#85	}
#86
#87	# v2 models - latest improvements
#88	config = {
#89	"reranker": {
#90	"provider": "huggingface",
#91	"config": {
#92	"model": "BAAI/bge-reranker-v2-m3",
#93	"device": "cuda"
#94	}
#95	}
#96	}
#97	```
#98
#99	### Multilingual Models
#100
#101	```python
#102	# Multilingual BGE reranker
#103	config = {
#104	"reranker": {
#105	"provider": "huggingface",
#106	"config": {
#107	"model": "BAAI/bge-reranker-v2-multilingual",
#108	"device": "cuda"
#109	}
#110	}
#111	}
#112	```
#113
#114	### Domain-Specific Models
#115
#116	```python
#117	# For code search
#118	config = {
#119	"reranker": {
#120	"provider": "huggingface",
#121	"config": {
#122	"model": "microsoft/codebert-base",
#123	"device": "cuda"
#124	}
#125	}
#126	}
#127
#128	# For biomedical content
#129	config = {
#130	"reranker": {
#131	"provider": "huggingface",
#132	"config": {
#133	"model": "dmis-lab/biobert-base-cased-v1.1",
#134	"device": "cuda"
#135	}
#136	}
#137	}
#138	```
#139
#140	## Usage Examples
#141
#142	### Basic Usage
#143
#144	```python
#145	from mem0 import Memory
#146
#147	m = Memory.from_config(config)
#148
#149	# Add some memories
#150	m.add("I love hiking in the mountains", user_id="alice")
#151	m.add("Pizza is my favorite food", user_id="alice")
#152	m.add("I enjoy reading science fiction books", user_id="alice")
#153
#154	# Search with reranking
#155	results = m.search(
#156	"What outdoor activities do I enjoy?",
#157	user_id="alice",
#158	rerank=True
#159	)
#160
#161	for result in results["results"]:
#162	print(f"Memory: {result['memory']}")
#163	print(f"Score: {result['score']:.3f}")
#164	```
#165
#166	### Batch Processing
#167
#168	```python
#169	# Process multiple queries efficiently
#170	queries = [
#171	"What are my hobbies?",
#172	"What food do I like?",
#173	"What books interest me?"
#174	]
#175
#176	results = []
#177	for query in queries:
#178	result = m.search(query, user_id="alice", rerank=True)
#179	results.append(result)
#180	```
#181
#182	## Performance Optimization
#183
#184	### GPU Acceleration
#185
#186	```python
#187	# Use GPU for better performance
#188	config = {
#189	"reranker": {
#190	"provider": "huggingface",
#191	"config": {
#192	"model": "BAAI/bge-reranker-base",
#193	"device": "cuda",
#194	"batch_size": 64, # Increase batch size for GPU
#195	}
#196	}
#197	}
#198	```
#199
#200	### Memory Optimization
#201
#202	```python
#203	# For limited memory environments
#204	config = {
#205	"reranker": {
#206	"provider": "huggingface",
#207	"config": {
#208	"model": "BAAI/bge-reranker-base",
#209	"device": "cpu",
#210	"batch_size": 8, # Smaller batch size
#211	"max_length": 256, # Shorter sequences
#212	"model_kwargs": {
#213	"torch_dtype": "float16" # Half precision
#214	}
#215	}
#216	}
#217	}
#218	```
#219
#220	## Model Comparison
#221
#222	\| Model \| Size \| Quality \| Speed \| Memory \| Best For \|
#223	\|-------\|------\|---------\|-------\|---------\|----------\|
#224	\| bge-reranker-base \| 278M \| Good \| Fast \| Low \| General use \|
#225	\| bge-reranker-large \| 560M \| Better \| Medium \| Medium \| High quality needs \|
#226	\| bge-reranker-v2-m3 \| 568M \| Best \| Medium \| Medium \| Latest improvements \|
#227	\| bge-reranker-v2-multilingual \| 568M \| Good \| Medium \| Medium \| Multiple languages \|
#228
#229	## Error Handling
#230
#231	```python
#232	try:
#233	results = m.search(
#234	"test query",
#235	user_id="alice",
#236	rerank=True
#237	)
#238	except Exception as e:
#239	print(f"Reranking failed: {e}")
#240	# Fall back to vector search only
#241	results = m.search(
#242	"test query",
#243	user_id="alice",
#244	rerank=False
#245	)
#246	```
#247
#248	## Custom Models
#249
#250	### Using Private Models
#251
#252	```python
#253	# Use a private model from Hugging Face
#254	config = {
#255	"reranker": {
#256	"provider": "huggingface",
#257	"config": {
#258	"model": "your-org/custom-reranker",
#259	"device": "cuda",
#260	"use_auth_token": "your-hf-token"
#261	}
#262	}
#263	}
#264	```
#265
#266	### Local Model Path
#267
#268	```python
#269	# Use a locally downloaded model
#270	config = {
#271	"reranker": {
#272	"provider": "huggingface",
#273	"config": {
#274	"model": "/path/to/local/model",
#275	"device": "cuda"
#276	}
#277	}
#278	}
#279	```
#280
#281	## Best Practices
#282
#283	1. Choose the Right Model: Balance quality vs speed based on your needs
#284	2. Use GPU: Significantly faster than CPU for larger models
#285	3. Optimize Batch Size: Tune based on your hardware capabilities
#286	4. Monitor Memory: Watch GPU/CPU memory usage with large models
#287	5. Cache Models: Download once and reuse to avoid repeated downloads
#288
#289	## Troubleshooting
#290
#291	### Common Issues
#292
#293	Out of Memory Error
#294	```python
#295	# Reduce batch size and sequence length
#296	config = {
#297	"reranker": {
#298	"provider": "huggingface",
#299	"config": {
#300	"model": "BAAI/bge-reranker-base",
#301	"batch_size": 4,
#302	"max_length": 256
#303	}
#304	}
#305	}
#306	```
#307
#308	Model Download Issues
#309	```python
#310	# Set cache directory
#311	import os
#312	os.environ["TRANSFORMERS_CACHE"] = "/path/to/cache"
#313
#314	# Or use offline mode
#315	config = {
#316	"reranker": {
#317	"provider": "huggingface",
#318	"config": {
#319	"model": "BAAI/bge-reranker-base",
#320	"local_files_only": True
#321	}
#322	}
#323	}
#324	```
#325
#326	CUDA Not Available
#327	```python
#328	import torch
#329
#330	config = {
#331	"reranker": {
#332	"provider": "huggingface",
#333	"config": {
#334	"model": "BAAI/bge-reranker-base",
#335	"device": "cuda" if torch.cuda.is_available() else "cpu"
#336	}
#337	}
#338	}
#339	```
#340
#341	## Next Steps
#342
#343	<CardGroup cols={2}>
#344	<Card title="Reranker Overview" icon="sort" href="/components/rerankers/overview">
#345	Learn about reranking concepts
#346	</Card>
#347	<Card title="Configuration Guide" icon="gear" href="/components/rerankers/config">
#348	Detailed configuration options
#349	</Card>
#350	</CardGroup>

z6Mkq5mY3JWtxoxUobWcfNHm7AkRubgSWEZTkBVqZXJviFZ5/my-project-public