uv.reporter package

Submodules

uv.reporter.base module

Base and Combinators; these classes and functions allow you to compose reporters together into compound reporters.

class uv.reporter.base.AbstractReporter

Bases: object

Base class for all reporters. A reporter is a type that is able to log timeseries of values for different t.MetricKey instances, one item at a time.

NOTE - by default, report_all and report are implemented in terms of one another. This means that you can choose which method you’d like to override, or override both… but if you don’t override any you’ll see infinite recursion.

Be careful not to abuse the kindness!

close() → None

Release any resources held open by this reporter instance.

filter_step(pred: Callable[[int], bool], on_false: Optional[AbstractReporter] = None)

Accepts a predicate function from step to boolean, and returns a reporter that tests every step against the supplied function. If the function returns true, metrics get passed on to this reporter; else, they get filtered out.

If a reporter is supplied to on_false, any time the predicate returns false items are routes to that store instead of base.

filter_values(pred: Callable[[int, Any], bool], on_false: Optional[AbstractReporter] = None)

“Accepts a function from (step, metric) to boolean; every (step, metric) pair passed to report and report_all are passed into this function. If the predicate returns true, the metric is passed on; else, it’s filtered.

from_thunk(thunk: Callable[], Dict[str, Any]])

Returns a new Reporter that passes all AbstractReporter methods through, but adds a new method called “thunk()” that, when called, will pass the emitted map of metric key to metric down to the underlying store.

thunk() returns the value emitted by the no-arg function passed here via thunk.

map_values(fn: Callable[[int, Any], Any])

“Accepts a function from (step, metric) to some new metric; every (step, metric) pair passed to report and report_all are passed into this function, and the result is passed down the chain to this, the calling reporter.

plus(*others: uv.reporter.base.AbstractReporter)

Returns an instance of MultiReporter wrapping the current instance. This reporter broadcasts its inputs to this instance, plus any other reporters supplied to this method, every time it sees a metric passed in via report or report_all.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_each_n(n: int)

Returns a new reporter that only reports every n steps; specifically, the new reporter will only accept metrics where step % n == 0.

If n <= 1, this reporter, untouched, is returned directly.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

stepped(step_key: Optional[str] = None)

Returns a new reporter that modifies incoming metrics by wrapping them in a dict of this form before passing them down to this instance of reporter:

{step_key: step, “value”: metric_value}

where step_key is the supplied argument, and equal to “step” by default. This is useful for keeping track of each metric’s timestamp.

with_prefix(prefix: Union[str, List[str]])

Returns an instance of PrefixedReporter wrapping the current instance. This reporter attaches the supplied prefix to every metric key it sees before passing metrics on.

with_suffix(suffix: Union[str, List[str]])

Returns an instance of SuffixedReporter wrapping the current instance. This reporter attaches the supplied suffix to every metric key it sees before passing metrics on.

class uv.reporter.base.FilterValuesReporter(base: uv.reporter.base.AbstractReporter, predicate: Callable[[int, Any], bool], on_false_reporter: Optional[uv.reporter.base.AbstractReporter] = None)

Bases: uv.reporter.base.AbstractReporter

Reporter that filters incoming metrics by applying a predicate from (step, t.Metric). If true, the reporter passes the result on to the underlying reporter. Else, nothing.

Parameters

• base – Backing reporter. All report and report_all calls proxy here.

• predicate – function from (step, metric) to metric.

close() → None

Release any resources held open by this reporter instance.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

class uv.reporter.base.MapValuesReporter(base: uv.reporter.base.AbstractReporter, fn: Callable[[int, Any], Any])

Bases: uv.reporter.base.AbstractReporter

Reporter that modifies incoming metrics by applying a function from (step, t.Metric) to a new metric before passing the result on to the underlying reporter.

Parameters

• base – Backing reporter. All report and report_all calls proxy here.

• fn – function from (step, metric) to metric.

close() → None

Release any resources held open by this reporter instance.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

class uv.reporter.base.MultiReporter(*reporters: uv.reporter.base.AbstractReporter)

Bases: uv.reporter.base.AbstractReporter

Reporter that broadcasts out metrics to all N reporters supplied to its constructor.

Parameters

reporters – instances of t.AbstractReporter that will receive all calls to this instance’s methods.

close() → None

Release any resources held open by this reporter instance.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

class uv.reporter.base.PrefixedReporter(base: uv.reporter.base.AbstractReporter, prefix: Union[str, List[str]])

Bases: uv.reporter.base.AbstractReporter

Reporter that prepends a prefix to all keys before passing requests on to the supplied backing reporter.

Parameters

• base – Backing reporter. All report and report_all calls proxy here.

• prefix – the prefix to attach to all keys supplied to any method.

close() → None

Release any resources held open by this reporter instance.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

class uv.reporter.base.SuffixedReporter(base: uv.reporter.base.AbstractReporter, suffix: Union[str, List[str]])

Bases: uv.reporter.base.AbstractReporter

Reporter that prepends a prefix to all keys before passing requests on to the supplied backing reporter.

Parameters

• base – Backing reporter. All report and report_all calls proxy here.

• suffix – the suffix to attach to all keys supplied to any method.

close() → None

Release any resources held open by this reporter instance.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

class uv.reporter.base.ThunkReporter(base: uv.reporter.base.AbstractReporter, thunk)

Bases: uv.reporter.base.AbstractReporter

Reporter that passes all AbstractReporter methods through, but adds a new method called “thunk()” that, when called, will pass the emitted map of metric key to metric down to the underlying store.

Parameters

• base – Backing reporter. All report and report_all calls proxy here.

• thunk – no-arg lambda that returns a metric dictionary.

close() → None

Release any resources held open by this reporter instance.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

thunk(step: int) → None

uv.reporter.base.stepped_reporter(base: uv.reporter.base.AbstractReporter, step_key: Optional[str] = None) → uv.reporter.base.AbstractReporter

Returns a new reporter that modifies incoming metric by wrapping them in a dict of this form before passing them down to base:

{step_key: step, “value”: metric_value}

where step_key is the supplied argument, and equal to “step” by default. This is useful for keeping track of each metric’s timestamp.

uv.reporter.state module

Reporter interface and implementations.

uv.reporter.state.active_reporter(r: uv.reporter.base.AbstractReporter)

uv.reporter.state.get_reporter() → uv.reporter.base.AbstractReporter

Returns the active reporter set using set_reporter()

uv.reporter.state.report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into the globally available reporter returned by uv.get_reporter().

uv.reporter.state.report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into the globally available reporter returned by uv.get_reporter().

uv.reporter.state.report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics. Reports to the globally available reporter returned by uv.get_reporter().

uv.reporter.state.report_params(m: Dict[str, str]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics. Reports to the globally available reporter returned by uv.get_reporter().

uv.reporter.state.set_reporter(r: uv.reporter.base.AbstractReporter) → uv.reporter.base.AbstractReporter

Set the globally available reporter instance. Returns its input.

uv.reporter.state.start_run(param_prefix: Optional[str] = None, experiment_name: Optional[str] = None, run_name: Optional[str] = None, artifact_location: Optional[str] = None, **args) → mlflow.tracking.fluent.ActiveRun

Close alias of mlflow.start_run. The only difference is that uv.start_run attempts to extract parameters from the environment and log those to the bound UV reporter using report_params.

Note that if experiment_name is specified and refers to an existing experiment, then the artifact_location will not be honored as this is an immutable property of an mlflow experiment. This method will issue a warning but proceed.

Note that the returned value can be used as a context manager: https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.start_run

uv.reporter.store module

AbstractReporter implementations that live at the bottom of the reporter stack. These reporters aren’t combinators; they’re responsible for persisting metrics into underlying store or mechanism.

class uv.reporter.store.LambdaReporter(report: Optional[Callable[[int, str, Any], None]] = None, report_all: Optional[Callable[[int, Dict[str, Any]], None]] = None, report_param: Optional[Callable[[str, str], None]] = None, report_params: Optional[Callable[[Dict[str, str]], None]] = None, close: Optional[Callable[[], None]] = None)

Bases: uv.reporter.base.AbstractReporter

AbstractReporter implementation that defers to a supplied lambda for its persistence ability. This allows you to escape the object-oriented programming paradigm, if you so choose.

Parameters

• report – Function called whenever reporter.report(step, k, v) is called.

• report_all – Function called whenever reporter.report_all(step, m) is called.

• report_param – Function called whenever reporter.report_param(k, v) is called.

• report_params – Function called whenever reporter.report_params(m) is called.

• close – If supplied, this no-arg function will get called by this instance’s close method.

close() → None

Release any resources held open by this reporter instance.

report(step, k, v)

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step, m)

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, str]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

class uv.reporter.store.LoggingReporter(file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, digits: int = 3)

Bases: uv.reporter.base.AbstractReporter

Reporter that logs all data to the file handle you pass in using a fairly sane format. Compatible with tqdm, the python progress bar.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

static tqdm()

Returns a logging reporter that will work with a tqdm progress bar.

class uv.reporter.store.MemoryReporter(m: Optional[Dict[str, List[Any]]] = None, params_store: Optional[Dict[str, str]] = None)

Bases: uv.reporter.base.AbstractReporter

Reporter that stores metrics in a Python dictionary, keyed by t.MetricKey. Metrics are stored as a list.

Parameters

m – Optional dictionary mapping metric keys to a list of accumulated metric values. If supplied, this dictionary will be mutated as new metrics arrive.

clear()

Erase all key-value pairs in the backing store.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, str]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

class uv.reporter.store.NullReporter

Bases: uv.reporter.base.AbstractReporter

Reporter that does nothing with the metrics passed to its various methods. reader() returns an instance of rs.EmptyReader.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step, k, v)

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step, m)

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

Module contents

Reporter interface and implementations.

class uv.reporter.AbstractReporter

Bases: object

Base class for all reporters. A reporter is a type that is able to log timeseries of values for different t.MetricKey instances, one item at a time.

NOTE - by default, report_all and report are implemented in terms of one another. This means that you can choose which method you’d like to override, or override both… but if you don’t override any you’ll see infinite recursion.

Be careful not to abuse the kindness!

close() → None

Release any resources held open by this reporter instance.

filter_step(pred: Callable[[int], bool], on_false: Optional[AbstractReporter] = None)

Accepts a predicate function from step to boolean, and returns a reporter that tests every step against the supplied function. If the function returns true, metrics get passed on to this reporter; else, they get filtered out.

If a reporter is supplied to on_false, any time the predicate returns false items are routes to that store instead of base.

filter_values(pred: Callable[[int, Any], bool], on_false: Optional[AbstractReporter] = None)

“Accepts a function from (step, metric) to boolean; every (step, metric) pair passed to report and report_all are passed into this function. If the predicate returns true, the metric is passed on; else, it’s filtered.

from_thunk(thunk: Callable[], Dict[str, Any]])

Returns a new Reporter that passes all AbstractReporter methods through, but adds a new method called “thunk()” that, when called, will pass the emitted map of metric key to metric down to the underlying store.

thunk() returns the value emitted by the no-arg function passed here via thunk.

map_values(fn: Callable[[int, Any], Any])

“Accepts a function from (step, metric) to some new metric; every (step, metric) pair passed to report and report_all are passed into this function, and the result is passed down the chain to this, the calling reporter.

plus(*others: uv.reporter.base.AbstractReporter)

Returns an instance of MultiReporter wrapping the current instance. This reporter broadcasts its inputs to this instance, plus any other reporters supplied to this method, every time it sees a metric passed in via report or report_all.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report(step: int, k: str, v: Any) → None

Accepts a step (an ordered int referencing some timestep), a metric key and a value, and persists the metric into some underlying store.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_each_n(n: int)

Returns a new reporter that only reports every n steps; specifically, the new reporter will only accept metrics where step % n == 0.

If n <= 1, this reporter, untouched, is returned directly.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, Union[str, Dict]]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.

stepped(step_key: Optional[str] = None)

Returns a new reporter that modifies incoming metrics by wrapping them in a dict of this form before passing them down to this instance of reporter:

{step_key: step, “value”: metric_value}

where step_key is the supplied argument, and equal to “step” by default. This is useful for keeping track of each metric’s timestamp.

with_prefix(prefix: Union[str, List[str]])

Returns an instance of PrefixedReporter wrapping the current instance. This reporter attaches the supplied prefix to every metric key it sees before passing metrics on.

with_suffix(suffix: Union[str, List[str]])

Returns an instance of SuffixedReporter wrapping the current instance. This reporter attaches the supplied suffix to every metric key it sees before passing metrics on.

class uv.reporter.LoggingReporter(file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, digits: int = 3)

Bases: uv.reporter.base.AbstractReporter

Reporter that logs all data to the file handle you pass in using a fairly sane format. Compatible with tqdm, the python progress bar.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

static tqdm()

Returns a logging reporter that will work with a tqdm progress bar.

class uv.reporter.MemoryReporter(m: Optional[Dict[str, List[Any]]] = None, params_store: Optional[Dict[str, str]] = None)

Bases: uv.reporter.base.AbstractReporter

Reporter that stores metrics in a Python dictionary, keyed by t.MetricKey. Metrics are stored as a list.

Parameters

m – Optional dictionary mapping metric keys to a list of accumulated metric values. If supplied, this dictionary will be mutated as new metrics arrive.

clear()

Erase all key-value pairs in the backing store.

reader() → Optional[uv.reader.base.AbstractReader]

Returns an implementation of AbstractReader that can access the data in this store.

Returns None by default; extending classes are encouraged, but not required, to override.

report_all(step: int, m: Dict[str, Any]) → None

Accepts a step (an ordered int referencing some timestep) and a dictionary of metric key => metric value, and persists the metric into some underlying store.

Extending classes are expected to perform some side effect that’s either visually useful, as in a live-plot, or recoverable via some matching extension of AbstractReader.

report_param(k: str, v: str) → None

Accepts a key and value parameter and logs these as parameters alongside the reported metrics.

report_params(m: Dict[str, str]) → None

Accepts a dict of parameter name -> value, and logs these as parameters alongside the reported metrics.