repositories
loading repo index
repositories
loading repo index
repository
loading code, commits, and activity
public Clawd ADK gateway launch mirror
stars
latest
clone command
git clone gitlawb://did:key:z6Mkq5mY...iFZ5/my-project-publ...git clone gitlawb://did:key:z6Mkq5mY.../my-project-publ...2fa351d6docs: add automaton and perps launch sources16d ago| #1 | --- |
| #2 | title: '📝 evaluate' |
| #3 | --- |
| #4 | |
| #5 | `evaluate()` method is used to evaluate the performance of a RAG app. You can find the signature below: |
| #6 | |
| #7 | ### Parameters |
| #8 | |
| #9 | <ParamField path="question" type="Union[str, list[str]]"> |
| #10 | A question or a list of questions to evaluate your app on. |
| #11 | </ParamField> |
| #12 | <ParamField path="metrics" type="Optional[list[Union[BaseMetric, str]]]" optional> |
| #13 | The metrics to evaluate your app on. Defaults to all metrics: `["context_relevancy", "answer_relevancy", "groundedness"]` |
| #14 | </ParamField> |
| #15 | <ParamField path="num_workers" type="int" optional> |
| #16 | Specify the number of threads to use for parallel processing. |
| #17 | </ParamField> |
| #18 | |
| #19 | ### Returns |
| #20 | |
| #21 | <ResponseField name="metrics" type="dict"> |
| #22 | Returns the metrics you have chosen to evaluate your app on as a dictionary. |
| #23 | </ResponseField> |
| #24 | |
| #25 | ## Usage |
| #26 | |
| #27 | ```python |
| #28 | from embedchain import App |
| #29 | |
| #30 | app = App() |
| #31 | |
| #32 | # add data source |
| #33 | app.add("https://www.forbes.com/profile/elon-musk") |
| #34 | |
| #35 | # run evaluation |
| #36 | app.evaluate("what is the net worth of Elon Musk?") |
| #37 | # {'answer_relevancy': 0.958019958036268, 'context_relevancy': 0.12903225806451613} |
| #38 | |
| #39 | # or |
| #40 | # app.evaluate(["what is the net worth of Elon Musk?", "which companies does Elon Musk own?"]) |
| #41 | ``` |
| #42 |