Python internal API

mobu.asyncio Module

asyncio utility functions for mobu.

Functions

schedule_periodic(func, interval)	Schedule a function to run periodically.
wait_first(*args)	Return the result of the first awaitable to finish.

Classes

aclosing_iter(thing)

Automatically close async iterators that are generators.

Class Inheritance Diagram

mobu.config Module

Configuration definition.

Classes

Config	Configuration for mobu.
GitHubCiAppConfig	Configuration for GitHub CI app functionality if it is enabled.
GitHubRefreshAppConfig	Configuration for GitHub refresh app functionality.

Class Inheritance Diagram

mobu.constants Module

Global constants for mobu.

Variables

CONFIGURATION_PATH	Default path to configuration.
GITHUB_REPO_CONFIG_PATH	The path to a config file with repo-specific configuration.
GITHUB_WEBHOOK_WAIT_SECONDS	GithHub needs some time to actually be in the state in a webhook payload.
NOTEBOOK_REPO_BRANCH	Default repository branch for NotebookRunner.
NOTEBOOK_REPO_URL	Default notebook repository for NotebookRunner.
TOKEN_LIFETIME	Token lifetime for mobu's service tokens.
USERNAME_REGEX	Regex matching all valid usernames.
WEBSOCKET_OPEN_TIMEOUT	How long to wait for a WebSocket connection to open (in seconds).

mobu.events Module

App metrics events.

Classes

EmptyLoopExecution	Reported when an empty loop.
EventBase	Attributes on every mobu event.
Events()	Container for app metrics event publishers.
GitLfsCheck	Reported from Git LFS businesses.
NotebookBase	Attributes for all notebook-related events.
NotebookCellExecution	Reported after a notebook cell is finished executing.
NotebookExecution	Reported after a notebook is finished executing.
NubladoDeleteLab	Reported for every attempt to delete a lab.
NubladoPythonExecution	Reported after a nublado python execution.
NubladoSpawnLab	Reported for every attempt to spawn a lab.
SIAQuery	Reported when a SIA query is executed.
TapQuery	Reported when a TAP query is executed.

Class Inheritance Diagram

mobu.exceptions Module

Exceptions for mobu.

Classes

ComparisonError(*, expected, received)	Comparing two strings failed.
FlockNotFoundError(flock)	The named flock was not found.
GafaelfawrParseError(message, error[, user])	Unable to parse the reply from Gafaelfawr.
GafaelfawrWebError(message, *[, method, ...])	An API call to Gafaelfawr failed.
GitHubFileNotFoundError	Tried to retrieve contents for a non-existent file in a GitHub repo.
MonkeyNotFoundError(monkey)	The named monkey was not found.
NotRetainingLogsError()	Mobu is not configured to retain logs.
SIAClientError(exc)	Creating an SIA client failed.
SubprocessError(msg, *[, returncode, ...])	Running a subprocess failed.

Class Inheritance Diagram

mobu.factory Module

Component factory and process-wide status for mobu.

Classes

Factory(context[, logger])	Component factory for mobu.
ProcessContext(http_client, events)	Per-process application context.

Class Inheritance Diagram

mobu.main Module

The main application factory for the mobu service.

Functions

create_app(*[, load_config])	Create the FastAPI application.
lifespan(app)	Set up and tear down the the base application.

mobu.sentry Module

Helpers for sentry instrumentation.

Functions

before_send(event, hint)	Add tags to fingerprint so that distinct issues are created.
capturing_start_span(op, **kwargs)	Start a span, set the op/start time in the context, and capture errors.
fingerprint(event)	Generate a fingerprint to force separate issues for tag combos.
start_transaction(name, op, **kwargs)	Start a transaction and mark it if an exception is raised.

mobu.status Module

Post periodic status to Slack.

Functions

post_status()

Post a summary of mobu status to Slack.

mobu.dependencies.config Module

Config dependency.

Classes

ConfigDependency([path])

Dependency to manage a cached Mobu configuration.

Variables

config_dependency

The dependency that will return the global configuration.

Class Inheritance Diagram

mobu.dependencies.context Module

Request context dependency for FastAPI.

This dependency gathers a variety of information into a single object for the convenience of writing request handlers. It also provides a place to store a structlog.BoundLogger that can gather additional context during processing, including from dependencies.

Functions

anonymous_context_dependency(request, logger)

Per-request context for non-gafaelfawr-auth'd requests.

Classes

ContextDependency()	Provide a per-request context as a FastAPI dependency.
RequestContext(request, logger, manager, ...)	Holds the incoming request and its surrounding context.

Variables

context_dependency

The dependency that will return the per-request context.

Class Inheritance Diagram

mobu.dependencies.github Module

Dependencies GitHub CI app functionality.

Classes

CiManagerDependency()	A process-global object to manage background CI workers.
MaybeCiManagerDependency(dep)	Try to return a CiManager, but don't blow up if it's not there.

Class Inheritance Diagram

mobu.handlers.external Module

Handlers for the app’s external root, /mobu/.

Variables

external_router

FastAPI router for all external handlers.

mobu.handlers.github_ci_app Module

Github webhook handlers for CI app.

Variables

api_router

Registers incoming HTTP GitHub webhook requests

mobu.handlers.github_refresh_app Module

Github webhook handlers for CI app.

Variables

api_router

Registers incoming HTTP GitHub webhook requests

mobu.handlers.internal Module

Internal HTTP handlers that serve relative to the root path, /.

These handlers aren’t externally visible since the app is available at a path, /mobu. See mobu.handlers.external for the external endpoint handlers.

These handlers should be used for monitoring, health checks, internal status, or other information that should not be visible outside the Kubernetes cluster.

Variables

internal_router

FastAPI router for all internal handlers.

mobu.models.ci_manager Module

Models for CiManager.

Classes

CiJobSummary	Information about a job.
CiManagerSummary	Information about the CiManager.
CiWorkerSummary	Information about a running worker.

Class Inheritance Diagram

mobu.models.flock Module

Models for a collection of monkeys.

Classes

FlockConfig	Configuration for a flock of monkeys.
FlockData	Information about a running flock.
FlockSummary	Summary statistics about a running flock.

Class Inheritance Diagram

mobu.models.index Module

Data models for the top-level route.

Classes

Index

Metadata returned by the external root URL of the application.

Class Inheritance Diagram

mobu.models.monkey Module

Data models for a monkey.

Classes

MonkeyData	Data for a running monkey.
MonkeyState(value[, names, module, ...])	State of a running monkey.

Class Inheritance Diagram

mobu.models.repo Module

Models related to GitHub repos for the GitHub CI app functionality.

Classes

ClonedRepoInfo(dir, path, hash)	Information about a cloned git repo.
RepoConfig	In-repo configuration for mobu behavior.

Class Inheritance Diagram

mobu.models.solitary Module

Models for running a single instance of a business by itself.

Classes

SolitaryConfig	Configuration for a solitary monkey.
SolitaryResult	Results from executing a solitary monkey.

Class Inheritance Diagram

mobu.models.summary Module

Combined summary of different functionalities.

Classes

CombinedSummary

Summary of all app state.

Class Inheritance Diagram

mobu.models.user Module

Data models for an authenticated user.

Classes

AuthenticatedUser	Represents an authenticated user with a token.
Group	Configuration for the group memberships of a user.
User	Configuration for the user whose credentials the monkey will use.
UserSpec	Configuration to generate a set of users.

Class Inheritance Diagram

mobu.models.business.base Module

Base models for monkey business.

Classes

BusinessConfig	Base configuration class for monkey business.
BusinessData	Status of a running business.
BusinessOptions	Options for monkey business.

Class Inheritance Diagram

mobu.models.business.business_config_type Module

Business config type helpers.

Variables

BusinessConfigType

A union type alias of all of all busines config types.

mobu.models.business.empty Module

Models for the EmptyLoop business.

Classes

EmptyLoopConfig

Configuration specialization for EmptyLoop.

Class Inheritance Diagram

mobu.models.business.gitlfs Module

Base models for Git-LFS-related monkey business.

Classes

GitLFSBusinessOptions	Options for business that runs git LFS operations.
GitLFSConfig	Configuration specialization for GitLFS.

Class Inheritance Diagram

mobu.models.business.notebookrunner Module

Shared models for different notebook runners.

Classes

CollectionRule	A set of patterns to filter the list of notebooks to run in a repo.
Filterable	Mixin for config to specify patterns for which notebooks to run.
NotebookFilterResults	Valid notebooks and categories for invalid notebooks.
NotebookMetadata	Notebook metadata that we care about.
NotebookRunnerData	Status of a running NotebookRunner business.
NotebookRunnerOptions	Options for all types NotebookRunner monkey business.

Class Inheritance Diagram

mobu.models.business.notebookrunnercounting Module

Models for the NotebookRunnerCounting monkey business.

Classes

NotebookRunnerCountingConfig	Configuration specialization for NotebookRunnerCounting.
NotebookRunnerCountingOptions	Options to specify a fixed number of notebooks to run per session.

Class Inheritance Diagram

mobu.models.business.notebookrunnerinfinite Module

Models for the NotebookRunnerInfinite monkey business.

Classes

NotebookRunnerInfiniteConfig

Configuration specialization for NotebookRunner.

Class Inheritance Diagram

mobu.models.business.notebookrunnerlist Module

Models for the NotebookRunnerList monkey business.

Classes

NotebookRunnerListConfig

Configuration specialization for NotebookRunnerList.

Class Inheritance Diagram

mobu.models.business.nublado Module

Base models for Nublado-related monkey business.

Classes

NubladoBusinessData	Status of a running Nublado business.
NubladoBusinessOptions	Options for any business that runs code in a Nublado lab.
RunningImage	Information about the running Jupyter lab image.

Class Inheritance Diagram

mobu.models.business.nubladopythonloop Module

Models for the NubladoPythonLoop monkey business.

Classes

NubladoPythonLoopConfig	Configuration specialization for NubladoPythonLoop.
NubladoPythonLoopOptions	Options for NubladoPythonLoop monkey business.

Class Inheritance Diagram

mobu.models.business.siaquerysetrunner Module

Models for the SIAQuerySetRunner monkey business.

Classes

SIABusinessData	Status of a running SIA business.
SIAQuery	The parameters of an SIA (v2) query.
SIAQuerySetRunnerConfig	Configuration specialization for SIAQuerySetRunner.
SIAQuerySetRunnerOptions	Options for SIAQuerySetRunner monkey business.

Class Inheritance Diagram

mobu.models.business.tap Module

Base models for TAP-related monkey business.

Classes

TAPBusinessData	Status of a running TAPQueryRunner business.
TAPBusinessOptions	Options for any business that runs TAP queries.

Class Inheritance Diagram

mobu.models.business.tapqueryrunner Module

Models for the TAPQueryRunner monkey business.

Classes

TAPQueryRunnerConfig	Configuration specialization for TAPQueryRunner.
TAPQueryRunnerOptions	Options for TAPQueryRunner monkey business.

Class Inheritance Diagram

mobu.models.business.tapquerysetrunner Module

Models for the TAPQuerySetRunner monkey business.

Classes

TAPQuerySetRunnerConfig	Configuration specialization for TAPQuerySetRunner.
TAPQuerySetRunnerOptions	Options for TAPQueryRunner monkey business.

Class Inheritance Diagram

mobu.services.flock Module

A flock of monkeys doing business.

Classes

Flock(*, flock_config, replica_count, ...)

Container for a group of monkeys all running the same business.

Class Inheritance Diagram

mobu.services.manager Module

Manager for all the running flocks.

Classes

FlockManager(*, gafaelfawr_storage, ...)

Manages all of the running flocks.

Class Inheritance Diagram

mobu.services.monkey Module

The monkey.

Classes

Monkey(*, name[, flock])

Runs one business and manages its log and configuration.

Class Inheritance Diagram

mobu.services.notebook_finder Module

Helpers to pick which notebooks in a repo to execute.

Classes

NotebookFinder(*, repo_path, repo_config[, ...])

A helper to select which notebooks to execute based on config.

Class Inheritance Diagram

mobu.services.repo Module

Helpers for cloning and filtering notebook repos.

Classes

RepoManager(logger, *[, testing])

A reference-counting caching repo cloner.

Class Inheritance Diagram

mobu.services.solitary Module

Manager for a solitary monkey.

Classes

Solitary(*, solitary_config, ...)

Runs a single monkey to completion and reports its results.

Class Inheritance Diagram

mobu.services.business.base Module

Base class for business logic for mobu.

Classes

Business(*, options, user, events, logger, flock)	Base class for monkey business (one type of repeated operation).
BusinessCommand(value[, names, module, ...])	Commands sent over the internal control queue.
CommonEventAttrs	Attributes on every event.

Class Inheritance Diagram

mobu.services.business.empty Module

EmptyLoop business logic for mobu.

Classes

EmptyLoop(*, options, user, events, logger, ...)

Business class that does nothing, successfully.

Class Inheritance Diagram

mobu.services.business.gitlfs Module

Class for executing Git-LFS tests.

Classes

GitLFSBusiness(*, options, user, events, ...)

Test a Git-LFS service.

Class Inheritance Diagram

mobu.services.business.notebookrunner Module

NotebookRunner logic for mobu.

This business pattern will clone a Git repo full of notebooks, iterate through the notebooks, and run them on the remote Nublado lab.

Classes

ExecutionIteration(iterator, size)	Properties of a set of notebook executions.
NotebookRunner(*, options, user, ...)	Start a Jupyter lab and run a sequence of notebooks.

Class Inheritance Diagram

mobu.services.business.notebookrunnercounting Module

Execute notebooks in batches of a fixed number per JuypterLab session.

Classes

NotebookRunnerCounting(*, options, user, ...)

A notebook runner that refreshes JupyterLab sessions (and optionally deletes the labs) after a configurable number of notebook executions.

Class Inheritance Diagram

mobu.services.business.notebookrunnerinfinite Module

Execute all notebooks in a single JupyterLab session.

Classes

NotebookRunnerInfinite(*, options, user, ...)

A notebook runner that never refreshes JupyterLab sessions or deletes labs until mobu is shut down.

Class Inheritance Diagram

mobu.services.business.notebookrunnerlist Module

Execute a list of notebooks in a single JupyterLab session.

Classes

NotebookRunnerList(*, options, user, ...)

A notebook runner that refreshes JupyterLab sessions (and optionally deletes the labs) after a list of notebooks has been executed.

Class Inheritance Diagram

mobu.services.business.nublado Module

Base class for executing code in a Nublado notebook.

Classes

NubladoBusiness(*, options, user, events, ...)	Base class for business that executes Python code in a Nublado notebook.
ProgressLogMessage(message[, timestamp])	A single log message with timestamp from spawn progress.

Class Inheritance Diagram

mobu.services.business.nubladopythonloop Module

NubladoPythonLoop logic for mobu.

This business pattern will start a lab and run some code in a loop over and over again.

Classes

NubladoPythonLoop(*, options, user, events, ...)

Run simple Python code in a loop inside a lab kernel.

Class Inheritance Diagram

mobu.services.business.siaquerysetrunner Module

Run a set of predefined queries against a SIA service.

Classes

SIAQuerySetRunner(*, options, user, events, ...)

Run queries from a predefined set against SIA with random parameters.

Class Inheritance Diagram

mobu.services.business.tap Module

Base class for executing TAP queries.

Classes

TAPBusiness(*, options, user, events, ...)

Base class for business that executes TAP query.

Class Inheritance Diagram

mobu.services.business.tapqueryrunner Module

Run queries against a TAP service.

Classes

TAPQueryRunner(*, options, user, events, ...)

Run queries against TAP.

Class Inheritance Diagram

mobu.services.business.tapquerysetrunner Module

Run a set of predefined queries against a TAP service.

Classes

TAPQuerySetRunner(*, options, user, events, ...)

Run queries from a predefined set against TAP with random parameters.

Class Inheritance Diagram

mobu.services.github_ci.ci_manager Module

Manager for background workers that process work from GitHub checks.

Classes

CiManager(github_app_id, github_private_key, ...)	Manages processing work for GitHub CI checks.
JobLifecycle([processing, processed])	States that a CI job can be in.

Class Inheritance Diagram

mobu.services.github_ci.ci_notebook_job Module

GitHub CI checks for notebook repos.

Classes

CiNotebookJob(github_storage, check_run, ...)

Runs changed notebooks and updates a GitHub CI check.

Class Inheritance Diagram

mobu.storage.gafaelfawr Module

Manage Gafaelfawr users and tokens.

Classes

GafaelfawrStorage(http_client, logger)

Manage users and authentication tokens.

Class Inheritance Diagram

mobu.storage.git Module

A very simple async replacement for GitPython.

GitPython is not thread- or async-safe because of a git quirk: current working directory is a per-process global, and “git add” requires that the cwd be inside a worktree.

Classes

Git(*[, repo, config_location, logger])	A very basic async Git client based on asyncio.subprocess.
Output(stdout, stderr)	The output from running a command in a subprocess.

Class Inheritance Diagram

mobu.storage.github Module

Tools for interacting with the GitHub REST API.

Classes

CheckRun(github_storage, id, name, summary)	Manage GitHub check runs via the GitHub API.
GitHubStorage(client, repo_owner, repo_name, ...)	Tools to interact with the GitHub API.

Class Inheritance Diagram