API Overview

`class` `Agent`

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
model_name: <class 'str'>
temperature: <class 'float'>
system_message: <class 'str'>
tools: list[typing.Any]

`method` `step`

step(state: AgentState) → AgentState

Run a step of the agent. Args:

state: The current state of the environment.
action: The action to take. Returns: The new state of the environment.

Source

`class` `AgentState`

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
history: list[typing.Any]

Source

`class` `AnnotationSpec`

Pydantic Fields:

name: str | None
description: str | None
field_schema: dict[str, typing.Any]
unique_among_creators: <class 'bool'>
op_scope: list[str] | None

Source

`classmethod` `preprocess_field_schema`

preprocess_field_schema(data: dict[str, Any]) → dict[str, Any]

Source

`classmethod` `validate_field_schema`

validate_field_schema(schema: dict[str, Any]) → dict[str, Any]

Source

`method` `value_is_valid`

value_is_valid(payload: Any) → bool

Validates a payload against this annotation spec’s schema. Args:

payload: The data to validate against the schema Returns:
bool: True if validation succeeds, False otherwise

Source

`class` `Audio`

A class representing audio data in a supported format (wav or mp3). This class handles audio data storage and provides methods for loading from different sources and exporting to files. Attributes:

format: The audio format (currently supports ‘wav’ or ‘mp3’)
data: The raw audio data as bytes

Args:

data: The audio data (bytes or base64 encoded string)
format: The audio format (‘wav’ or ‘mp3’)
validate_base64: Whether to attempt base64 decoding of the input data Raises:
ValueError: If audio data is empty or format is not supported

Source

`method` `init`

__init__(
    data: 'bytes',
    format: 'SUPPORTED_FORMATS_TYPE',
    validate_base64: 'bool' = True
) → None

Source

`method` `export`

export(path: 'str | bytes | Path | PathLike') → None

Export audio data to a file. Args:

Source

`classmethod` `from_data`

from_data(data: 'str | bytes', format: 'str') → Self

Create an Audio object from raw data and specified format.

path: Path where the audio file should be written Args:
data: Audio data as bytes or base64 encoded string
format: Audio format (‘wav’ or ‘mp3’) Returns:
Audio: A new Audio instance

Raises:

ValueError: If format is not supported

Source

`classmethod` `from_path`

from_path(path: 'str | bytes | Path | PathLike') → Self

Create an Audio object from a file path. Args:

path: Path to an audio file (must have .wav or .mp3 extension) Returns:
Audio: A new Audio instance loaded from the file

Raises:

ValueError: If file doesn’t exist or has unsupported extension

Source

`class` `Content`

A class to represent content from various sources, resolving them to a unified byte-oriented representation with associated metadata. This class must be instantiated using one of its classmethods:

from_path()
from_bytes()
from_text()
from_url()
from_base64()
from_data_url()

Source

`method` `init`

__init__(*args: 'Any', **kwargs: 'Any') → None

Direct initialization is disabled. Please use a classmethod like Content.from_path() to create an instance. Pydantic Fields:

data: <class 'bytes'>
size: <class 'int'>
mimetype: <class 'str'>
digest: <class 'str'>
filename: <class 'str'>
content_type: typing.Literal['bytes', 'text', 'base64', 'file', 'url', 'data_url', 'data_url:base64', 'data_url:encoding', 'data_url:encoding:base64']
input_type: <class 'str'>
encoding: <class 'str'>
metadata: dict[str, typing.Any] | None
extension: str | None

`property` art

`property` ref

Source

`method` `as_string`

as_string() → str

Display the data as a string. Bytes are decoded using the encoding attribute If base64, the data will be re-encoded to base64 bytes then decoded to an ASCII string Returns: str.

Source

`classmethod` `from_base64`

from_base64(
    b64_data: 'str | bytes',
    extension: 'str | None' = None,
    mimetype: 'str | None' = None,
    metadata: 'dict[str, Any] | None' = None
) → Self

Initializes Content from a base64 encoded string or bytes.

Source

`classmethod` `from_bytes`

from_bytes(
    data: 'bytes',
    extension: 'str | None' = None,
    mimetype: 'str | None' = None,
    metadata: 'dict[str, Any] | None' = None,
    encoding: 'str' = 'utf-8'
) → Self

Initializes Content from raw bytes.

Source

`classmethod` `from_data_url`

from_data_url(url: 'str', metadata: 'dict[str, Any] | None' = None) → Self

Initializes Content from a data URL.

Source

`classmethod` `from_path`

from_path(
    path: 'str | Path',
    encoding: 'str' = 'utf-8',
    mimetype: 'str | None' = None,
    metadata: 'dict[str, Any] | None' = None
) → Self

Initializes Content from a local file path.

Source

`classmethod` `from_text`

from_text(
    text: 'str',
    extension: 'str | None' = None,
    mimetype: 'str | None' = None,
    metadata: 'dict[str, Any] | None' = None,
    encoding: 'str' = 'utf-8'
) → Self

Initializes Content from a string of text.

Source

`classmethod` `from_url`

from_url(
    url: 'str',
    headers: 'dict[str, Any] | None' = None,
    timeout: 'int | None' = 30,
    metadata: 'dict[str, Any] | None' = None
) → Self

Initializes Content by fetching bytes from an HTTP(S) URL. Downloads the content, infers mimetype/extension from headers, URL path, and data, and constructs a Content object from the resulting bytes.

Source

`classmethod` `model_validate`

model_validate(
    obj: 'Any',
    strict: 'bool | None' = None,
    from_attributes: 'bool | None' = None,
    context: 'dict[str, Any] | None' = None
) → Self

Override model_validate to handle Content reconstruction from dict.

Source

`classmethod` `model_validate_json`

model_validate_json(
    json_data: 'str | bytes | bytearray',
    strict: 'bool | None' = None,
    context: 'dict[str, Any] | None' = None
) → Self

Override model_validate_json to handle Content reconstruction from JSON.

Source

`method` `open`

open() → bool

Open the file using the operating system’s default application. This method uses the platform-specific mechanism to open the file with the default application associated with the file’s type. Returns:

bool: True if the file was successfully opened, False otherwise.

Source

`method` `save`

save(dest: 'str | Path') → None

Copy the file to the specified destination path. Updates the filename and the path of the content to reflect the last saved copy. Args:

Source

`method` `serialize_data`

serialize_data(data: 'bytes') → str

When dumping model in json mode

Source

`method` `to_data_url`

to_data_url(use_base64: 'bool' = True) → str

Constructs a data URL from the content.

dest: Destination path where the file will be copied to (string or pathlib.Path) The destination path can be a file or a directory. If dest has no file extension (e.g. .txt), destination will be considered a directory. Args:
use_base64: If True, the data will be base64 encoded. Otherwise, it will be percent-encoded. Defaults to True. Returns: A data URL string.

Source

`class` `Dataset`

Dataset object with easy saving and automatic versioning. Examples:

# Create a dataset
dataset = Dataset(name='grammar', rows=[
     {'id': '0', 'sentence': "He no likes ice cream.", 'correction': "He doesn't like ice cream."},
     {'id': '1', 'sentence': "She goed to the store.", 'correction': "She went to the store."},
     {'id': '2', 'sentence': "They plays video games all day.", 'correction': "They play video games all day."}
])

# Publish the dataset
weave.publish(dataset)

# Retrieve the dataset
dataset_ref = weave.ref('grammar').get()

# Access a specific example
example_label = dataset_ref.rows[2]['sentence']

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
rows: trace.table.Table | trace.vals.WeaveTable

Source

`method` `add_rows`

add_rows(rows: Iterable[dict]) → Dataset

Create a new dataset version by appending rows to the existing dataset. This is useful for adding examples to large datasets without having to load the entire dataset into memory. Args:

rows: The rows to add to the dataset. Returns: The updated dataset.

Source

`classmethod` `convert_to_table`

convert_to_table(rows: Any) → Table | WeaveTable

Source

`classmethod` `from_calls`

from_calls(calls: Iterable[Call]) → Self

Source

`classmethod` `from_hf`

from_hf(
    hf_dataset: Union[ForwardRef('HFDataset'), ForwardRef('HFDatasetDict')]
) → Self

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`classmethod` `from_pandas`

from_pandas(df: 'DataFrame') → Self

Source

`method` `select`

select(indices: Iterable[int]) → Self

Select rows from the dataset based on the provided indices. Args:

indices: An iterable of integer indices specifying which rows to select. Returns: A new Dataset object containing only the selected rows.

Source

`method` `to_hf`

to_hf() → HFDataset

Source

`method` `to_pandas`

to_pandas() → DataFrame

Source

`class` `EasyPrompt`

Source

`method` `init`

__init__(
    content: str | dict | list | None = None,
    role: str | None = None,
    dedent: bool = False,
    **kwargs: Any
) → None

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
data: <class 'list'>
config: <class 'dict'>
requirements: <class 'dict'>

`property` as_str

Join all messages into a single string.

`property` is_bound

`property` messages

`property` placeholders

`property` system_message

Join all messages into a system prompt message.

`property` system_prompt

Join all messages into a system prompt object.

`property` unbound_placeholders

Source

`method` `append`

append(item: Any, role: str | None = None, dedent: bool = False) → None

Source

`method` `as_dict`

as_dict() → dict[str, Any]

Source

`method` `as_pydantic_dict`

as_pydantic_dict() → dict[str, Any]

Source

`method` `bind`

bind(*args: Any, **kwargs: Any) → Prompt

Source

`method` `bind_rows`

bind_rows(dataset: list[dict] | Any) → list['Prompt']

Source

`method` `config_table`

config_table(title: str | None = None) → Table

Source

`method` `configure`

configure(config: dict | None = None, **kwargs: Any) → Prompt

Source

`method` `dump`

dump(fp: <class 'IO'>) → None

Source

`method` `dump_file`

dump_file(filepath: str | Path) → None

Source

`method` `format`

format(**kwargs: Any) → Any

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`classmethod` `load`

load(fp: <class 'IO'>) → Self

Source

`classmethod` `load_file`

load_file(filepath: str | Path) → Self

Source

`method` `messages_table`

messages_table(title: str | None = None) → Table

Source

`method` `print`

print() → str

Source

`method` `publish`

publish(name: str | None = None) → ObjectRef

Source

`method` `require`

require(param_name: str, **kwargs: Any) → Prompt

Source

`method` `run`

run() → Any

Source

`method` `validate_requirement`

validate_requirement(key: str, value: Any) → list

Source

`method` `validate_requirements`

validate_requirements(values: dict[str, Any]) → list

Source

`method` `values_table`

values_table(title: str | None = None) → Table

Source

`class` `Evaluation`

Sets up an evaluation which includes a set of scorers and a dataset. Calling evaluation.evaluate(model) will pass in rows from a dataset into a model matching the names of the columns of the dataset to the argument names in model.predict. Then it will call all of the scorers and save the results in weave. If you want to preprocess the rows from the dataset you can pass in a function to preprocess_model_input. Examples:

# Collect your examples
examples = [
     {"question": "What is the capital of France?", "expected": "Paris"},
     {"question": "Who wrote 'To Kill a Mockingbird'?", "expected": "Harper Lee"},
     {"question": "What is the square root of 64?", "expected": "8"},
]

# Define any custom scoring function
@weave.op
def match_score1(expected: str, model_output: dict) -> dict:
     # Here is where you'd define the logic to score the model output
     return {'match': expected == model_output['generated_text']}

@weave.op
def function_to_evaluate(question: str):
     # here's where you would add your LLM call and return the output
     return  {'generated_text': 'Paris'}

# Score your examples using scoring functions
evaluation = Evaluation(
     dataset=examples, scorers=[match_score1]
)

# Start tracking the evaluation
weave.init('intro-example')
# Run the evaluation
asyncio.run(evaluation.evaluate(function_to_evaluate))

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
dataset: <class 'dataset.dataset.Dataset'>
scorers: list[typing.Annotated[trace.op_protocol.Op | flow.scorer.Scorer, BeforeValidator(func=<function cast_to_scorer at 0x7fd95dcf47c0>, json_schema_input_type=PydanticUndefined)]] | None
preprocess_model_input: collections.abc.Callable[[dict], dict] | None
trials: <class 'int'>
metadata: dict[str, typing.Any] | None
evaluation_name: str | collections.abc.Callable[trace.call.Call, str] | None

Source

`method` `evaluate`

evaluate(model: Op | Model) → dict

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`method` `get_eval_results`

get_eval_results(model: Op | Model) → EvaluationResults

Source

`method` `get_evaluate_calls`

get_evaluate_calls() → PaginatedIterator[CallSchema, WeaveObject]

Retrieve all evaluation calls that used this Evaluation object. Note that this returns a CallsIter instead of a single call because it’s possible to have multiple evaluation calls for a single evaluation (e.g. if you run the same evaluation multiple times). Returns:

CallsIter: An iterator over Call objects representing evaluation runs.

Raises:

ValueError: If the evaluation has no ref (hasn’t been saved/run yet).

Examples:

evaluation = Evaluation(dataset=examples, scorers=[scorer])
await evaluation.evaluate(model)  # Run evaluation first
calls = evaluation.get_evaluate_calls()
for call in calls:
     print(f"Evaluation run: {call.id} at {call.started_at}")

Source

`method` `get_score_calls`

get_score_calls() → dict[str, list[Call]]

Retrieve scorer calls for each evaluation run, grouped by trace ID. Returns:

dict[str, list[Call]]: A dictionary mapping trace IDs to lists of scorer Call objects. Each trace ID represents one evaluation run, and the list contains all scorer calls executed during that run.

Examples:

evaluation = Evaluation(dataset=examples, scorers=[accuracy_scorer, f1_scorer])
await evaluation.evaluate(model)
score_calls = evaluation.get_score_calls()
for trace_id, calls in score_calls.items():
     print(f"Trace {trace_id}: {len(calls)} scorer calls")
     for call in calls:
         scorer_name = call.summary.get("weave", {}).get("trace_name")
         print(f"  Scorer: {scorer_name}, Output: {call.output}")

Source

`method` `get_scores`

get_scores() → dict[str, dict[str, list[Any]]]

Extract and organize scorer outputs from evaluation runs. Returns:

dict[str, dict[str, list[Any]]]: A nested dictionary structure where:
- First level keys are trace IDs (evaluation runs)
- Second level keys are scorer names
- Values are lists of scorer outputs for that run and scorer

Examples:

evaluation = Evaluation(dataset=examples, scorers=[accuracy_scorer, f1_scorer])
await evaluation.evaluate(model)
scores = evaluation.get_scores()
# Access scores by trace and scorer
for trace_id, trace_scores in scores.items():
         print(f"Evaluation run {trace_id}:")
         for scorer_name, outputs in trace_scores.items():
             print(f"  {scorer_name}: {outputs}")

Expected output:

{
     "trace_123": {
     "accuracy_scorer": [{"accuracy": 0.85}],
     "f1_scorer": [{"f1": 0.78}]
     }
}

Source

`method` `model_post_init`

model_post_init(_Evaluation__context: Any) → None

Source

`method` `predict_and_score`

predict_and_score(model: Op | Model, example: dict) → dict

Source

`method` `summarize`

summarize(eval_table: EvaluationResults) → dict

Source

`class` `EvaluationLogger`

This class provides an imperative interface for logging evaluations. An evaluation is started automatically when the first prediction is logged using the log_prediction method, and finished when the log_summary method is called. Each time you log a prediction, you will get back a ScoreLogger object. You can use this object to log scores and metadata for that specific prediction. For more information, see the ScoreLogger class. Basic usage - log predictions with inputs and outputs directly:

ev = EvaluationLogger()

# Log predictions with known inputs/outputs
pred = ev.log_prediction(inputs={'q': 'Hello'}, outputs={'a': 'Hi there!'})
pred.log_score("correctness", 0.9)

# Finish the evaluation
ev.log_summary({"avg_score": 0.9})

Advanced usage - use context manager for dynamic outputs and nested operations:

ev = EvaluationLogger()

# Use context manager when you need to capture nested operations
with ev.log_prediction(inputs={'q': 'Hello'}) as pred:
     # Any operations here (like LLM calls) automatically become
     # children of the predict call
     response = your_llm_call(...)
     pred.output = response.content
     pred.log_score("correctness", 0.9)

# Finish the evaluation
ev.log_summary({"avg_score": 0.9})

Source

`method` `init`

__init__(
    name: 'str | None' = None,
    model: 'Model | dict | str | None' = None,
    dataset: 'Dataset | list[dict] | str | None' = None,
    eval_attributes: 'dict[str, Any] | None' = None,
    scorers: 'list[str] | None' = None
) → None

`property` attributes

`property` ui_url

Source

`method` `fail`

fail(exception: 'BaseException') → None

Convenience method to fail the evaluation with an exception.

Source

`method` `finish`

finish(exception: 'BaseException | None' = None) → None

Clean up the evaluation resources explicitly without logging a summary. Ensures all prediction calls and the main evaluation call are finalized. This is automatically called if the logger is used as a context manager.

Source

`method` `log_example`

log_example(
    inputs: 'dict[str, Any]',
    output: 'Any',
    scores: 'dict[str, ScoreType]'
) → None

Log a complete example with inputs, output, and scores. This is a convenience method that combines log_prediction and log_score for when you have all the data upfront. Args:

inputs: The input data for the prediction
output: The output value
scores: Dictionary mapping scorer names to score values Example:

ev = EvaluationLogger()
ev.log_example(
    inputs={'q': 'What is 2+2?'},
    output='4',
    scores={'correctness': 1.0, 'fluency': 0.9}
)

Source

`method` `log_prediction`

log_prediction(inputs: 'dict[str, Any]', output: 'Any' = None) → ScoreLogger

Log a prediction to the Evaluation. Returns a ScoreLogger that can be used directly or as a context manager. Args:

inputs: The input data for the prediction
output: The output value. Defaults to None. Can be set later using pred.output. Returns: ScoreLogger for logging scores and optionally finishing the prediction.

Example (direct):

pred = ev.log_prediction({'q': ’…’}, output=“answer”) pred.log_score(“correctness”, 0.9) pred.finish()

Example (context manager):

with ev.log_prediction({'q': ’…’}) as pred: response = model(…) pred.output = response pred.log_score(“correctness”, 0.9) # Automatically calls finish() on exit

Source

`method` `log_summary`

log_summary(summary: 'dict | None' = None, auto_summarize: 'bool' = True) → None

Log a summary dict to the Evaluation. This will calculate the summary, call the summarize op, and then finalize the evaluation, meaning no more predictions or scores can be logged.

Source

`method` `set_view`

set_view(
    name: 'str',
    content: 'Content | str',
    extension: 'str | None' = None,
    mimetype: 'str | None' = None,
    metadata: 'dict[str, Any] | None' = None,
    encoding: 'str' = 'utf-8'
) → None

Attach a view to the evaluation’s main call summary under weave.views. Saves the provided content as an object in the project and writes its reference URI under summary.weave.views.<name> for the evaluation’s evaluate call. String inputs are wrapped as text content using Content.from_text with the provided extension or mimetype. Args:

name: The view name to display, used as the key under summary.weave.views.
content: A weave.Content instance or string to serialize.
extension: Optional file extension for string content inputs.
mimetype: Optional MIME type for string content inputs.
metadata: Optional metadata attached to newly created Content.
encoding: Text encoding for string content inputs. Returns: None

Examples: import weave

ev = weave.EvaluationLogger() ev.set_view(“report”, ”# Report”, extension=“md”)

Source

`class` `File`

A class representing a file with path, mimetype, and size information. Source

`method` `init`

__init__(path: 'str | Path', mimetype: 'str | None' = None)

Initialize a File object. Args:

`property` filename

Get the filename of the file.

path: Path to the file (string or pathlib.Path)
mimetype: Optional MIME type of the file - will be inferred from extension if not provided Returns:
str: The name of the file without the directory path.

Source

`method` `open`

open() → bool

bool: True if the file was successfully opened, False otherwise.

Source

`method` `save`

save(dest: 'str | Path') → None

Copy the file to the specified destination path. Args:

Source

`class` `Markdown`

A Markdown renderable.

dest: Destination path where the file will be copied to (string or pathlib.Path) The destination path can be a file or a directory. Args:
markup (str): A string containing markdown.
code_theme (str, optional): Pygments theme for code blocks. Defaults to “monokai”. See https://pygments.org/styles/ for code themes.
justify (JustifyMethod, optional): Justify value for paragraphs. Defaults to None.
style (Union[str, Style], optional): Optional style to apply to markdown.
hyperlinks (bool, optional): Enable hyperlinks. Defaults to True.

Source

`method` `init`

__init__(
    markup: 'str',
    code_theme: 'str' = 'monokai',
    justify: 'JustifyMethod | None' = None,
    style: 'str | Style' = 'none',
    hyperlinks: 'bool' = True,
    inline_code_lexer: 'str | None' = None,
    inline_code_theme: 'str | None' = None
) → None

Source

`class` `MessagesPrompt`

Source

`method` `init`

__init__(messages: list[dict])

inline_code_lexer: (str, optional): Lexer to use if inline code highlighting is enabled. Defaults to None.
inline_code_theme: (Optional[str], optional): Pygments theme for inline code highlighting, or None for no highlighting. Defaults to None. Pydantic Fields:
name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
messages: list[dict]

Source

`method` `format`

format(**kwargs: Any) → list

Source

`method` `format_message`

format_message(message: dict, **kwargs: Any) → dict

Format a single message by replacing template variables. This method delegates to the standalone format_message_with_template_vars function for the actual formatting logic.

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`class` `Model`

Intended to capture a combination of code and data the operates on an input. For example it might call an LLM with a prompt to make a prediction or generate text. When you change the attributes or the code that defines your model, these changes will be logged and the version will be updated. This ensures that you can compare the predictions across different versions of your model. Use this to iterate on prompts or to try the latest LLM and compare predictions across different settings Examples:

class YourModel(Model):
     attribute1: str
     attribute2: int

     @weave.op
     def predict(self, input_data: str) -> dict:
         # Model logic goes here
         prediction = self.attribute1 + ' ' + input_data
         return {'pred': prediction}

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None

Source

`method` `get_infer_method`

get_infer_method() → Callable

Source

`class` `Monitor`

Sets up a monitor to score incoming calls automatically. Examples:

import weave
from weave.scorers import ValidJSONScorer

json_scorer = ValidJSONScorer()

my_monitor = weave.Monitor(
     name="my-monitor",
     description="This is a test monitor",
     sampling_rate=0.5,
     op_names=["my_op"],
     query={
         "$expr": {
             "$gt": [
                 {
                         "$getField": "started_at"
                     },
                     {
                         "$literal": 1742540400
                     }
                 ]
             }
         }
     },
     scorers=[json_scorer],
)

my_monitor.activate()

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
sampling_rate: <class 'float'>
scorers: list[flow.scorer.Scorer]
op_names: list[str]
query: trace_server.interface.query.Query | None
active: <class 'bool'>

Source

`method` `activate`

activate() → ObjectRef

Activates the monitor. Returns: The ref to the monitor.

Source

`method` `deactivate`

deactivate() → ObjectRef

Deactivates the monitor. Returns: The ref to the monitor.

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`class` `Object`

Base class for Weave objects that can be tracked and versioned. This class extends Pydantic’s BaseModel to provide Weave-specific functionality for object tracking, referencing, and serialization. Objects can have names, descriptions, and references that allow them to be stored and retrieved from the Weave system. Attributes:

name (Optional[str]): A human-readable name for the object.
description (Optional[str]): A description of what the object represents.
ref (Optional[ObjectRef]): A reference to the object in the Weave system.

Examples:

# Create a simple object
obj = Object(name="my_object", description="A test object")

# Create an object from a URI
obj = Object.from_uri("weave:///entity/project/object:digest")

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None

Source

`classmethod` `from_uri`

from_uri(uri: str, objectify: bool = True) → Self

Create an object instance from a Weave URI. Args:

uri (str): The Weave URI pointing to the object.
objectify (bool): Whether to objectify the result. Defaults to True.

Returns:

Self: An instance of the class created from the URI.

Raises:

NotImplementedError: If the class doesn’t implement the required methods for deserialization.

Examples:

obj = MyObject.from_uri("weave:///entity/project/object:digest")

Source

`classmethod` `handle_relocatable_object`

handle_relocatable_object(
    v: Any,
    handler: ValidatorFunctionWrapHandler,
    info: ValidationInfo
) → Any

Handle validation of relocatable objects including ObjectRef and WeaveObject. This validator handles special cases where the input is an ObjectRef or WeaveObject that needs to be properly converted to a standard Object instance. It ensures that references are preserved and that ignored types are handled correctly during the validation process. Args:

v (Any): The value to validate.
handler (ValidatorFunctionWrapHandler): The standard pydantic validation handler.
info (ValidationInfo): Validation context information.

Returns:

Any: The validated object instance.

Examples: This method is called automatically during object creation and validation. It handles cases like: ```python

When an ObjectRef is passed

obj = MyObject(some_object_ref)

When a WeaveObject is passed

obj = MyObject(some_weave_object)

---

<SourceLink url="https://github.com/wandb/weave/blob/v0.52.24/weave/trace/refs.py#L159" />

## <kbd>class</kbd> `ObjectRef`
ObjectRef(entity: 'str', project: 'str', name: 'str', _digest: 'str | Future[str]', _extra: 'tuple[str | Future[str], ...]' = ()) 

<SourceLink url="https://github.com/wandb/weave/blob/v0.52.24/../../../../weave/trace/refs/__init__" />

### <kbd>method</kbd> `__init__`

```python
__init__(
entity: 'str',
project: 'str',
name: 'str',
_digest: 'str | Future[str]',
_extra: 'tuple[str | Future[str], ]' = ()
) → None

`property` digest

`property` extra

Source

`method` `as_param_dict`

as_param_dict() → dict

Source

`method` `delete`

delete() → None

Source

`method` `get`

get(objectify: 'bool' = True) → Any

Source

`method` `is_descended_from`

is_descended_from(potential_ancestor: 'ObjectRef') → bool

Source

`method` `maybe_parse_uri`

maybe_parse_uri(s: 'str') → AnyRef | None

Source

`method` `parse_uri`

parse_uri(uri: 'str') → ObjectRef

Source

`method` `uri`

uri() → str

Source

`method` `with_attr`

with_attr(attr: 'str') → Self

Source

`method` `with_extra`

with_extra(extra: 'tuple[str | Future[str], ]') → Self

Source

`method` `with_index`

with_index(index: 'int') → Self

Source

`method` `with_item`

with_item(item_digest: 'str | Future[str]') → Self

Source

`method` `with_key`

with_key(key: 'str') → Self

Source

`class` `EasyPrompt`

Source

`method` `init`

__init__(
    content: str | dict | list | None = None,
    role: str | None = None,
    dedent: bool = False,
    **kwargs: Any
) → None

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
data: <class 'list'>
config: <class 'dict'>
requirements: <class 'dict'>

`property` as_str

Join all messages into a single string.

`property` is_bound

`property` messages

`property` placeholders

`property` system_message

Join all messages into a system prompt message.

`property` system_prompt

Join all messages into a system prompt object.

`property` unbound_placeholders

Source

`method` `append`

append(item: Any, role: str | None = None, dedent: bool = False) → None

Source

`method` `as_dict`

as_dict() → dict[str, Any]

Source

`method` `as_pydantic_dict`

as_pydantic_dict() → dict[str, Any]

Source

`method` `bind`

bind(*args: Any, **kwargs: Any) → Prompt

Source

`method` `bind_rows`

bind_rows(dataset: list[dict] | Any) → list['Prompt']

Source

`method` `config_table`

config_table(title: str | None = None) → Table

Source

`method` `configure`

configure(config: dict | None = None, **kwargs: Any) → Prompt

Source

`method` `dump`

dump(fp: <class 'IO'>) → None

Source

`method` `dump_file`

dump_file(filepath: str | Path) → None

Source

`method` `format`

format(**kwargs: Any) → Any

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`classmethod` `load`

load(fp: <class 'IO'>) → Self

Source

`classmethod` `load_file`

load_file(filepath: str | Path) → Self

Source

`method` `messages_table`

messages_table(title: str | None = None) → Table

Source

`method` `print`

print() → str

Source

`method` `publish`

publish(name: str | None = None) → ObjectRef

Source

`method` `require`

require(param_name: str, **kwargs: Any) → Prompt

Source

`method` `run`

run() → Any

Source

`method` `validate_requirement`

validate_requirement(key: str, value: Any) → list

Source

`method` `validate_requirements`

validate_requirements(values: dict[str, Any]) → list

Source

`method` `values_table`

values_table(title: str | None = None) → Table

Source

`class` `Prompt`

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None

Source

`method` `format`

format(**kwargs: Any) → Any

Source

`class` `SavedView`

A fluent-style class for working with SavedView objects. Source

`method` `init`

__init__(view_type: 'str' = 'traces', label: 'str' = 'SavedView') → None

`property` entity

`property` label

`property` project

`property` view_type

Source

`method` `add_column`

add_column(path: 'str | ObjectPath', label: 'str | None' = None) → SavedView

Source

`method` `add_columns`

add_columns(*columns: 'str') → SavedView

Convenience method for adding multiple columns to the grid.

Source

`method` `add_filter`

add_filter(
    field: 'str',
    operator: 'str',
    value: 'Any | None' = None
) → SavedView

Source

`method` `add_sort`

add_sort(field: 'str', direction: 'SortDirection') → SavedView

Source

`method` `column_index`

column_index(path: 'int | str | ObjectPath') → int

Source

`method` `filter_op`

filter_op(op_name: 'str | None') → SavedView

Source

`method` `get_calls`

get_calls(
    limit: 'int | None' = None,
    offset: 'int | None' = None,
    include_costs: 'bool' = False,
    include_feedback: 'bool' = False,
    all_columns: 'bool' = False
) → CallsIter

Get calls matching this saved view’s filters and settings.

Source

`method` `get_known_columns`

get_known_columns(num_calls_to_query: 'int | None' = None) → list[str]

Get the set of columns that are known to exist.

Source

`method` `get_table_columns`

get_table_columns() → list[TableColumn]

Source

`method` `hide_column`

hide_column(col_name: 'str') → SavedView

Source

`method` `insert_column`

insert_column(
    idx: 'int',
    path: 'str | ObjectPath',
    label: 'str | None' = None
) → SavedView

Source

`classmethod` `load`

load(ref: 'str') → Self

Source

`method` `page_size`

page_size(page_size: 'int') → SavedView

Source

`method` `pin_column_left`

pin_column_left(col_name: 'str') → SavedView

Source

`method` `pin_column_right`

pin_column_right(col_name: 'str') → SavedView

Source

`method` `remove_column`

remove_column(path: 'int | str | ObjectPath') → SavedView

Source

`method` `remove_columns`

remove_columns(*columns: 'str') → SavedView

Remove columns from the saved view.

Source

`method` `remove_filter`

remove_filter(index_or_field: 'int | str') → SavedView

Source

`method` `remove_filters`

remove_filters() → SavedView

Remove all filters from the saved view.

Source

`method` `rename`

rename(label: 'str') → SavedView

Source

`method` `rename_column`

rename_column(path: 'int | str | ObjectPath', label: 'str') → SavedView

Source

`method` `save`

save() → SavedView

Publish the saved view to the server.

Source

`method` `set_columns`

set_columns(*columns: 'str') → SavedView

Set the columns to be displayed in the grid.

Source

`method` `show_column`

show_column(col_name: 'str') → SavedView

Source

`method` `sort_by`

sort_by(field: 'str', direction: 'SortDirection') → SavedView

Source

`method` `to_grid`

to_grid(limit: 'int | None' = None) → Grid

Source

`method` `to_rich_table_str`

to_rich_table_str() → str

Source

`method` `ui_url`

ui_url() → str | None

URL to show this saved view in the UI. Note this is the “result” page with traces etc, not the URL for the view object.

Source

`method` `unpin_column`

unpin_column(col_name: 'str') → SavedView

Source

`class` `Scorer`

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
column_map: dict[str, str] | None

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`method` `model_post_init`

model_post_init(_Scorer__context: Any) → None

Source

`method` `score`

score(output: Any, **kwargs: Any) → Any

Source

`method` `summarize`

summarize(score_rows: list) → dict | None

Source

`class` `StringPrompt`

Source

`method` `init`

__init__(content: str)

Pydantic Fields:

name: str | None
description: str | None
ref: trace.refs.ObjectRef | None
content: <class 'str'>

Source

`method` `format`

format(**kwargs: Any) → str

Source

`classmethod` `from_obj`

from_obj(obj: WeaveObject) → Self

Source

`class` `Table`

Source

`method` `init`

__init__(rows: 'list[dict]') → None

`property` rows

Source

`method` `append`

append(row: 'dict') → None

Add a row to the table.

Source

`method` `pop`

pop(index: 'int') → None

Remove a row at the given index from the table.

Source

`class` `ContextAwareThread`

A Thread that runs functions with the context of the caller. This is a drop-in replacement for threading.Thread that ensures calls behave as expected inside the thread. Weave requires certain contextvars to be set (see call_context.py), but new threads do not automatically copy context from the parent, which can cause the call context to be lost — not good! This class automates contextvar copying so using this thread “just works” as the user probably expects. You can achieve the same effect without this class by instead writing:

def run_with_context(func, *args, **kwargs):
     context = copy_context()
     def wrapper():
         context.run(func, *args, **kwargs)
     return wrapper

thread = threading.Thread(target=run_with_context(your_func, *args, **kwargs))
thread.start()

Source

`method` `init`

__init__(*args: 'Any', **kwargs: 'Any') → None

`property` daemon

A boolean value indicating whether this thread is a daemon thread. This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False. The entire Python program exits when only daemon threads are left.

`property` ident

Thread identifier of this thread or None if it has not been started. This is a nonzero integer. See the get_ident() function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.

`property` name

A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.

`property` native_id

Native integral thread ID of this thread, or None if it has not been started. This is a non-negative integer. See the get_native_id() function. This represents the Thread ID as reported by the kernel.

Source

`method` `run`

run() → None

Source

`class` `ThreadContext`

Context object providing access to current thread and turn information. Source

`method` `init`

__init__(thread_id: 'str | None')

Initialize ThreadContext with the specified thread_id. Args:

`property` thread_id

Get the thread_id for this context.

thread_id: The thread identifier for this context, or None if disabled. Returns: The thread identifier, or None if thread tracking is disabled.

`property` turn_id

Get the current turn_id from the active context. Returns: The current turn_id if set, None otherwise.

Source

`class` `ContextAwareThreadPoolExecutor`

A ThreadPoolExecutor that runs functions with the context of the caller. This is a drop-in replacement for concurrent.futures.ThreadPoolExecutor that ensures weave calls behave as expected inside the executor. Weave requires certain contextvars to be set (see call_context.py), but new threads do not automatically copy context from the parent, which can cause the call context to be lost — not good! This class automates contextvar copying so using this executor “just works” as the user probably expects. You can achieve the same effect without this class by instead writing:

with concurrent.futures.ThreadPoolExecutor() as executor:
     contexts = [copy_context() for _ in range(len(vals))]

     def _wrapped_fn(*args):
         return contexts.pop().run(fn, *args)

     executor.map(_wrapped_fn, vals)

Source

`method` `init`

__init__(*args: 'Any', **kwargs: 'Any') → None

Source

`method` `map`

map(
    fn: 'Callable',
    *iterables: 'Iterable[Any]',
    timeout: 'float | None' = None,
    chunksize: 'int' = 1
) → Iterator

Source

`method` `submit`

submit(fn: 'Callable', *args: 'Any', **kwargs: 'Any') → Any

Source

`function` `as_op`

as_op(fn: 'Callable[P, R]') → Op[P, R]

Given a @weave.op decorated function, return its Op. @weave.op decorated functions are instances of Op already, so this function should be a no-op at runtime. But you can use it to satisfy type checkers if you need to access OpDef attributes in a typesafe way. Args:

fn: A weave.op decorated function. Returns: The Op of the function.

Source

`function` `attributes`

attributes(attributes: 'dict[str, Any]') → Iterator

Context manager for setting attributes on a call. Example:

with weave.attributes({'env': 'production'}):
     print(my_function.call("World"))

Source

`function` `finish`

finish() → None

Stops logging to weave. Following finish, calls of weave.op decorated functions will no longer be logged. You will need to run weave.init() again to resume logging.

Source

`function` `get`

get(uri: 'str | ObjectRef') → Any

A convenience function for getting an object from a URI. Many objects logged by Weave are automatically registered with the Weave server. This function allows you to retrieve those objects by their URI. Args:

uri: A fully-qualified weave ref URI. Returns: The object.

Example:

weave.init("weave_get_example")
dataset = weave.Dataset(rows=[{"a": 1, "b": 2}])
ref = weave.publish(dataset)

dataset2 = weave.get(ref)  # same as dataset!

Source

`function` `get_client`

get_client() → WeaveClient | None

Source

`function` `get_current_call`

get_current_call() → Call | None

Get the Call object for the currently executing Op, within that Op. Returns: The Call object for the currently executing Op, or None if tracking has not been initialized or this method is invoked outside an Op. Note:

The returned Call’s attributes dictionary becomes immutable once the call starts. Use :func:weave.attributes to set call metadata before invoking an Op. The summary field may be updated while the Op executes and will be merged with computed summary information when the call finishes.

Source

`function` `init`

init(
    project_name: 'str',
    settings: 'UserSettings | dict[str, Any] | None' = None,
    autopatch_settings: 'AutopatchSettings | None' = None,
    global_postprocess_inputs: 'PostprocessInputsFunc | None' = None,
    global_postprocess_output: 'PostprocessOutputFunc | None' = None,
    global_attributes: 'dict[str, Any] | None' = None
) → WeaveClient

Initialize weave tracking, logging to a wandb project. Logging is initialized globally, so you do not need to keep a reference to the return value of init. Following init, calls of weave.op decorated functions will be logged to the specified project. Args: NOTE: Global postprocessing settings are applied to all ops after each op’s own postprocessing. The order is always: 1. Op-specific postprocessing 2. Global postprocessing

project_name: The name of the Weights & Biases team and project to log to. If you don’t specify a team, your default entity is used. To find or update your default entity, refer to User Settings in the W&B Models documentation.
settings: Configuration for the Weave client generally.
autopatch_settings: (Deprecated) Configuration for autopatch integrations. Use explicit patching instead.
global_postprocess_inputs: A function that will be applied to all inputs of all ops.
global_postprocess_output: A function that will be applied to all outputs of all ops.
global_attributes: A dictionary of attributes that will be applied to all traces. Returns: A Weave client.

Source

`function` `log_call`

log_call(
    op: 'str',
    inputs: 'dict[str, Any]',
    output: 'Any',
    parent: 'Call | None' = None,
    attributes: 'dict[str, Any] | None' = None,
    display_name: 'str | Callable[[Call], str] | None' = None,
    use_stack: 'bool' = True,
    exception: 'BaseException | None' = None
) → Call

Log a call directly to Weave without using the decorator pattern. This function provides an imperative API for logging operations to Weave, useful when you want to log calls after they’ve already been executed or when the decorator pattern isn’t suitable for your use case. Args:

op (str): The operation name to log. This will be used as the op_name for the call. Anonymous operations (strings not referring to published ops) are supported.
inputs (dict[str, Any]): A dictionary of input parameters for the operation.
output (Any): The output/result of the operation.
parent (Call | None): Optional parent call to nest this call under. If not provided, the call will be a root-level call (or nested under the current call context if one exists). Defaults to None.
attributes (dict[str, Any] | None): Optional metadata to attach to the call. These are frozen once the call is created. Defaults to None.
display_name (str | Callable[[Call], str] | None): Optional display name for the call in the UI. Can be a string or a callable that takes the call and returns a string. Defaults to None.
use_stack (bool): Whether to push the call onto the runtime stack. When True, the call will be available in the call context and can be accessed via weave.require_current_call(). When False, the call is logged but not added to the call stack. Defaults to True.
exception (BaseException | None): Optional exception to log if the operation failed. Defaults to None.

Returns:

Call: The created and finished Call object with full trace information.

Examples: Basic usage:

import weave
    >>> weave.init('my-project')
    >>> call = weave.log_call(
    ...     op="my_function",
    ...     inputs={"x": 5, "y": 10},
    ...     output=15
    ... )

    Logging with attributes and display name:
    >>> call = weave.log_call(
    ...     op="process_data",
    ...     inputs={"data": [1, 2, 3]},
    ...     output={"mean": 2.0},
    ...     attributes={"version": "1.0", "env": "prod"},
    ...     display_name="Data Processing"
    ... )

    Logging a failed operation:
    >>> try:
    ...     result = risky_operation()
    ... except Exception as e:
    ...     call = weave.log_call(
    ...         op="risky_operation",
    ...         inputs={},
    ...         output=None,
    ...         exception=e
    ...     )

    Nesting calls:
    >>> parent_call = weave.log_call("parent", {"input": 1}, 2)
    >>> child_call = weave.log_call(
    ...     "child",
    ...     {"input": 2},
    ...     4,
    ...     parent=parent_call
    ... )

    Logging without adding to call stack:
    >>> call = weave.log_call(
    ...     op="background_task",
    ...     inputs={"task_id": 123},
    ...     output="completed",
    ...     use_stack=False  # Don't push to call stack
    ... )

---

<SourceLink url="https://github.com/wandb/weave/blob/v0.52.24/weave/trace/op.py#L1202" />

### <kbd>function</kbd> `op`

```python
op(
    func: 'Callable[P, R] | None' = None,
    name: 'str | None' = None,
    call_display_name: 'str | CallDisplayNameFunc | None' = None,
    postprocess_inputs: 'PostprocessInputsFunc | None' = None,
    postprocess_output: 'PostprocessOutputFunc | None' = None,
    tracing_sample_rate: 'float' = 1.0,
    enable_code_capture: 'bool' = True,
    accumulator: 'Callable[[Any | None, Any], Any] | None' = None,
    kind: 'OpKind | None' = None,
    color: 'OpColor | None' = None
) → Callable[[Callable[P, R]], Op[P, R]] | Op[P, R]

A decorator to weave op-ify a function or method. Works for both sync and async. Automatically detects iterator functions and applies appropriate behavior.

Source

`function` `publish`

publish(obj: 'Any', name: 'str | None' = None) → ObjectRef

Save and version a Python object. Weave creates a new version of the object if the object’s name already exists and its content hash does not match the latest version of that object. Args:

obj: The object to save and version.
name: The name to save the object under. Returns: A Weave Ref to the saved object.

Source

`function` `ref`

ref(location: 'str') → ObjectRef

Creates a Ref to an existing Weave object. This does not directly retrieve the object but allows you to pass it to other Weave API functions. Args:

location: A Weave Ref URI, or if weave.init() has been called, name:version or name. If no version is provided, latest is used. Returns: A Weave Ref to the object.

Source

`function` `require_current_call`

require_current_call() → Call

Get the Call object for the currently executing Op, within that Op. This allows you to access attributes of the Call such as its id or feedback while it is running.

@weave.op
def hello(name: str) -> None:
     print(f"Hello {name}!")
     current_call = weave.require_current_call()
     print(current_call.id)

It is also possible to access a Call after the Op has returned. If you have the Call’s id, perhaps from the UI, you can use the get_call method on the WeaveClient returned from weave.init to retrieve the Call object.

client = weave.init("<project>")
mycall = client.get_call("<call_id>")

Alternately, after defining your Op you can use its call method. For example:

@weave.op
def add(a: int, b: int) -> int:
     return a + b

result, call = add.call(1, 2)
print(call.id)

Returns: The Call object for the currently executing Op Raises:

NoCurrentCallError: If tracking has not been initialized or this method is invoked outside an Op.

Source

`function` `set_view`

set_view(
    name: 'str',
    content: 'Content | str',
    extension: 'str | None' = None,
    mimetype: 'str | None' = None,
    metadata: 'dict[str, Any] | None' = None,
    encoding: 'str' = 'utf-8'
) → None

Attach a custom view to the current call summary at _weave.views.<name>. Args:

name: The view name (key under summary._weave.views).
content: A weave.Content instance or raw string. Strings are wrapped via Content.from_text using the supplied extension or mimetype.
extension: Optional file extension to use when content is a string.
mimetype: Optional MIME type to use when content is a string.
metadata: Optional metadata to attach when creating Content from text.
encoding: Text encoding to apply when creating Content from text. Returns: None

Examples: import weave

weave.init(“proj”) @weave.op … def foo(): … weave.set_view(“readme”, ”# Hello”, extension=“md”) … return 1 foo()

Source

`function` `thread`

thread(
    thread_id: 'str | None | object' = <object object at 0x7fd95e3cc980>
) → Iterator[ThreadContext]

Context manager for setting thread_id on calls within the context. Examples:

# Auto-generate thread_id
with weave.thread() as t:
     print(f"Thread ID: {t.thread_id}")
     result = my_function("input")  # This call will have the auto-generated thread_id
     print(f"Current turn: {t.turn_id}")

# Explicit thread_id
with weave.thread("custom_thread") as t:
     result = my_function("input")  # This call will have thread_id="custom_thread"

# Disable threading
with weave.thread(None) as t:
     result = my_function("input")  # This call will have thread_id=None

Args:

thread_id: The thread identifier to associate with calls in this context. If not provided, a UUID v7 will be auto-generated. If None, thread tracking will be disabled. Yields:
ThreadContext: An object providing access to thread_id and current turn_id.

Source

`function` `wandb_init_hook`

wandb_init_hook() → None

Get Started

Guides

Cookbooks

Reference

Open Source

Community

​API Overview

​class Agent

​method step

​class AgentState

​class AnnotationSpec

​classmethod preprocess_field_schema

​classmethod validate_field_schema

​method value_is_valid

​class Audio

​method __init__

​method export

​classmethod from_data

​classmethod from_path

​class Content

​method __init__

​property art

​property ref

​method as_string

​classmethod from_base64

​classmethod from_bytes

​classmethod from_data_url

​classmethod from_path

​classmethod from_text

​classmethod from_url

​classmethod model_validate

​classmethod model_validate_json

​method open

​method save

​method serialize_data

​method to_data_url

​class Dataset

​method add_rows

​classmethod convert_to_table

​classmethod from_calls

​classmethod from_hf

​classmethod from_obj

​classmethod from_pandas

​method select

​method to_hf

​method to_pandas

​class EasyPrompt

​method __init__

​property as_str

​property is_bound

​property messages

​property placeholders

​property system_message

​property system_prompt

​property unbound_placeholders

​method append

​method as_dict

​method as_pydantic_dict

​method bind

​method bind_rows

​method config_table

​method configure

​method dump

​method dump_file

​method format

​classmethod from_obj

​classmethod load

​classmethod load_file

​method messages_table

​method print

​method publish

​method require

​method run

​method validate_requirement

​method validate_requirements

​method values_table

​class Evaluation

​method evaluate

​classmethod from_obj

​method get_eval_results

API Overview

`class` `Agent`

`method` `step`

`class` `AgentState`

`class` `AnnotationSpec`

`classmethod` `preprocess_field_schema`

`classmethod` `validate_field_schema`

`method` `value_is_valid`

`class` `Audio`

`method` `init`

`method` `export`

`classmethod` `from_data`

`classmethod` `from_path`

`class` `Content`

`method` `init`

`property` art

`property` ref

`method` `as_string`

`classmethod` `from_base64`

`classmethod` `from_bytes`

`classmethod` `from_data_url`

`classmethod` `from_path`

`classmethod` `from_text`

`classmethod` `from_url`

`classmethod` `model_validate`

`classmethod` `model_validate_json`

`method` `open`

`method` `save`

`method` `serialize_data`

`method` `to_data_url`

`class` `Dataset`

`method` `add_rows`

`classmethod` `convert_to_table`

`classmethod` `from_calls`

`classmethod` `from_hf`

`classmethod` `from_obj`

`classmethod` `from_pandas`

`method` `select`

`method` `to_hf`

`method` `to_pandas`

`class` `EasyPrompt`

`method` `init`

`property` as_str

`property` is_bound

`property` messages

`property` placeholders

`property` system_message

`property` system_prompt

`property` unbound_placeholders

`method` `append`

`method` `as_dict`

`method` `as_pydantic_dict`

`method` `bind`

`method` `bind_rows`

`method` `config_table`

`method` `configure`

`method` `dump`

`method` `dump_file`

`method` `format`

`classmethod` `from_obj`

`classmethod` `load`

`classmethod` `load_file`

`method` `messages_table`

`method` `print`

`method` `publish`

`method` `require`

`method` `run`

`method` `validate_requirement`

`method` `validate_requirements`

`method` `values_table`

`class` `Evaluation`

`method` `evaluate`

`classmethod` `from_obj`

`method` `get_eval_results`

`method` `get_evaluate_calls`

`method` `get_score_calls`

`method` `get_scores`

`method` `model_post_init`

`method` `predict_and_score`

`method` `summarize`