Statistics

These functions take an inference problem and its fit, and report the uncertainty and identifiability of the fit. See statistical estimation and identifiability.

Recommended defaults

After a fit, call information, laplace, identifiability and profile on the problem and its fit, and predict for bands. Run sample only after these look sound.

Local analysis

diff_soup.inference.information

information(problem: Any, *, at: Any, kind: str = 'observed') -> InformationResult

Assemble the information matrix, the curvature of the negative log likelihood, at a point.

For a MAP problem the result also holds the prior's curvature and their sum, the posterior precision. A multiple-shooting fit that fits its window starts has them integrated out, so the matrix covers the parameters alone.

Parameters:

Name	Type	Description	Default
problem	Any	An InferenceProblem whose measurements all declare a likelihood.	required
at	Any	The point, an OptResult, a parameter vector, or a mapping from every fitted handle to its value.	required
kind	str	"observed" for the curvature at the data seen, or "expected" for the curvature averaged over data drawn from the model.	'observed'

Returns:

Type	Description
InformationResult	An InformationResult holding the matrix, whose rank() counts the directions the data determine.

diff_soup.inference.laplace

laplace(problem: Any, fit: Any, *, curvature: str = 'observed', on_singular: str = 'raise')

Build the Laplace approximation, the Gaussian centered on a fit with the inverse curvature as covariance.

For a MAP problem the curvature includes the prior's, and log_evidence is the Laplace log evidence. A multiple-shooting fit that fits its window starts has them integrated out.

Parameters:

Name	Type	Description	Default
problem	Any	An InferenceProblem whose measurements all declare a likelihood.	required
fit	Any	The OptResult to center on, a parameter vector, or a mapping from every fitted handle to its value.	required
curvature	str	"observed" or "expected", as kind= of information.	'observed'
on_singular	str	What happens when the data leave a direction undetermined, "raise" to raise ImproperPosteriorError or "subspace" to return the determined directions.	'raise'

Returns:

Type	Description
	A LaplacePosterior indexed by handle, or an IdentifiedSubspace under on_singular="subspace" when a direction is undetermined.

diff_soup.inference.evidence

evidence(problem: Any, fit: Any, *, curvature: str = 'observed') -> EvidenceResult

Approximate the log marginal likelihood of the data at a MAP fit, by Laplace's method.

The value is log p(y | mode) + log p(mode) + d/2 log(2 pi) - 1/2 log det H, with H the posterior precision of ds.inference.laplace(problem, fit) and the priors the problem declares. A fit that is not at a stationary point is refused.

Parameters:

Name	Type	Description	Default
problem	Any	An InferenceProblem with objective=MAP(), whose measurements all declare a likelihood.	required
fit	Any	The OptResult of the MAP fit, a parameter vector, or a mapping from every fitted handle to its value.	required
curvature	str	"observed" or "expected", as curvature= of laplace.	'observed'

Returns:

Type	Description
EvidenceResult	An EvidenceResult whose log_evidence is the value and whose other fields are its terms.

diff_soup.inference.identifiability

identifiability(problem: EstimationProblem, fit: OptResult | Mapping, *, parameter_scales: Any = None, ridge: float = 1e-10, null_tol: float = 1e-10) -> IdentifiabilityReport

Find which parameter directions the data determine at a fit, and which they leave undetermined.

The prior and any regularization are left out, so neither can hide a direction the data leave open. The analysis is reported twice, in the parameters' own units and after scaling by parameter_scales. A multiple-shooting fit that fits its window starts has them integrated out.

Parameters:

Name	Type	Description	Default
problem	EstimationProblem	An InferenceProblem whose measurements all declare a likelihood.	required
fit	OptResult | Mapping	The OptResult of ds.inference.optsolve on problem, or a mapping from every fitted handle to its value.	required
parameter_scales	Any	Positive scales per fitted coordinate, as a vector or a mapping from handle or (handle, owner) to scale, or None for all ones.	None
ridge	float	A small value added to each eigenvalue before inverting it, for the correlations of the report only.	1e-10
null_tol	float	The eigenvalue, relative to the largest, at or below which a direction counts as undetermined.	1e-10

Returns:

Type	Description
IdentifiabilityReport	An IdentifiabilityReport holding the analysis in raw and scaled coordinates.

diff_soup.inference.profile

profile(problem: Any, fit: OptResult, handle: Any, *, owner=None, optimizer=None, levels=(0.95,), bounds=None, max_steps=24, multistart=0, multistart_scale=1.0)

Compute the profile likelihood of one scalar parameter, refitting every other parameter at each value.

Each side of the fit is walked until the profile passes the threshold of the largest level, a bound, or max_steps. A side that stays below the threshold is an open direction.

Parameters:

Name	Type	Description	Default
problem	Any	An InferenceProblem or LoweredStudy whose measurements all declare a likelihood.	required
fit	OptResult	The OptResult of ds.inference.optsolve on problem.	required
handle	Any	A scalar handle from spec.param.	required
owner		The member whose value is walked, required for a handle that varies by member and refused for a shared one.	None
optimizer		The optimizer of the refits, LevenbergMarquardt, or TrustRegion on a lowered study, or None for LevenbergMarquardt().	None
levels		The confidence levels whose likelihood-ratio thresholds the result uses.	(0.95,)
bounds		An optional range (lo, hi) for the walked value, which must contain the fit.	None
max_steps		The most accepted steps on each side.	24
multistart		The number of extra jittered starts per refit, of which the best is kept.	0
multistart_scale		The size of the jitter, relative to 1 + |value| of each coordinate.	1.0

Returns:

Type	Description
	A ProfileResult, whose interval(level) is the profile interval.

diff_soup.inference.residual_report

residual_report(problem, fit, *, experiment_id=None) -> ResidualReport

Standardize the residuals of each measurement by its likelihood at a fit, and flag departures from the noise model.

Under the declared noise model the standardized residuals have mean zero, standard deviation one, and no lag-1 autocorrelation.

Parameters:

Name	Type	Description	Default
problem		An InferenceProblem whose measurements all declare a likelihood.	required
fit		An OptResult, a parameter vector, or a mapping from every fitted handle to its value.	required
experiment_id		The experiment to check, required when the problem has several.	None

Returns:

Type	Description
ResidualReport	A ResidualReport with one entry per measurement in channels, and the departures in words in flags.

diff_soup.inference.practical_report

practical_report(problem: Any, fit: Any, *, posterior: Any = None, profiles: tuple = (), tolerances: Optional[dict] = None, report: Any = None, experiment_id: Any = None) -> PracticalReport

Collect the evidence on practical identifiability at a fit into one report.

It holds the information spectrum, the supplied profiles and posterior, and the residual report, and names the evidence that was not supplied.

Parameters:

Name	Type	Description	Default
problem	Any	An InferenceProblem whose measurements all declare a likelihood.	required
fit	Any	The OptResult of ds.inference.optsolve on problem.	required
posterior	Any	The result of laplace at the fit, whose standard errors enter the report.	None
profiles	tuple	Results of profile, whose intervals and open directions enter the report.	()
tolerances	Optional[dict]	A mapping from parameter name to the largest acceptable standard error, which needs posterior.	None
report	Any	The result of identifiability at this fit, or None to compute it.	None
experiment_id	Any	The experiment whose residuals are checked, required when the problem has several.	None

Returns:

Type	Description
PracticalReport	A PracticalReport, with each tolerance marked as met or not in status.

diff_soup.inference.function_report

function_report(information: Any, block: Any, *, metric: str = 'gram', basis: Any = None, gram: Any = None, nuisance: str = 'schur', scale: str = 'raw', at: Any = None, null_tol: float = 1e-10) -> FunctionReport

Report how well the data determine one function-valued parameter, measured in the function's own norm.

It takes the function's coefficients out of an information matrix and reports the eigenvalues of their information measured against metric.

Parameters:

Name	Type	Description	Default
information	Any	The result of information.	required
block	Any	The name of the function handle, or the column indices of its coefficients.	required
metric	str	"gram" to measure against the Gram matrix of the basis, or "prior" against the prior precision of the function.	'gram'
basis	Any	The function's basis, such as a Legendre, needed for the curves and used for the Gram matrix when gram is None.	None
gram	Any	The Gram matrix of the function's basis.	None
nuisance	str	"schur" to eliminate the other parameters, or "slice" to hold them fixed.	'schur'
scale	str	"raw" for curves of the raw function, or "natural" for curves through the basis's output transform.	'raw'
at	Any	The function's coefficient values, needed for scale="natural".	None
null_tol	float	The eigenvalue, relative to the largest, at or below which a mode counts as undetermined.	1e-10

Returns:

Type	Description
FunctionReport	A FunctionReport with the eigenvalues, the modes, and the undetermined null_functions.

diff_soup.inference.cluster_covariance

cluster_covariance(problem: EstimationProblem, fit: OptResult, *, cluster: str, null_tol: float = 1e-10) -> ClusterCovarianceResult

Estimate the cluster-robust sandwich covariance of a fit, with each cluster an independent sampling unit.

Rows within a cluster may be correlated, and the declared noise scale need not be right. For a MAP fit the result is the spread of the estimator, not a posterior.

Parameters:

Name	Type	Description	Default
problem	EstimationProblem	The InferenceProblem or LoweredStudy the fit was made on.	required
fit	OptResult	The OptResult of ds.inference.optsolve on problem.	required
cluster	str	The name of a cluster of the problem's study, with at least two distinct labels.	required
null_tol	float	The curvature, relative to the largest, at or below which a direction counts as undetermined, which raises ImproperPosteriorError.	1e-10

Returns:

Type	Description
ClusterCovarianceResult	A ClusterCovarianceResult holding the covariance, the cluster labels, and warnings such as too few clusters.

---

Bootstrap

diff_soup.inference.bootstrap

bootstrap(problem: Any, fit: Any, *, replicates: int, key: Any, over: Any = None, weights: str = 'bayesian', steps: Any = 3, weight_priors: bool = True, solver: Any = None, max_attempts: Any = None, require_converged: bool = True, checkpoint: Any = None, keep: str = 'last', every: int = 1, on_conflict: str = 'error', callbacks: Any = (), chunk: Any = 'auto', shard: Any = None) -> BootstrapResult

Bootstrap a hierarchical fit by reweighting the members of the study's top level.

Each replicate draws one weight per member and steps from the fit's point under those weights. The top level needs at least three members.

Parameters:

Name	Type	Description	Default
problem	Any	The LoweredStudy the fit was made on.	required
fit	Any	The OptResult of TrustRegion on problem.	required
replicates	int	The number of replicates, a positive integer.	required
key	Any	The JAX random key that fixes the weights, such as jax.random.key(0).	required
over	Any	The level to reweight, by name or as 1, which must be the top level, or None for the top level.	None
weights	str	"bayesian" for continuous weights that sum to the member count, or "multinomial" to resample the members with replacement.	'bayesian'
steps	Any	The accepted trust-region steps each replicate takes from the fit, 0 for a one-step estimate, or "refit" to refit each replicate to convergence.	3
weight_priors	bool	Whether each member's prior and regularization terms take the member's weight.	True
solver	Any	The TrustRegion the replicates step with, which must not set weights=, or None for TrustRegion(stationarity_tol=1e-8).	None
max_attempts	Any	The step attempts allowed per replicate, or None for the solver's max_steps under "refit" and 4 * steps otherwise.	None
require_converged	bool	Whether to refuse a fit that did not converge.	True
checkpoint	Any	A directory to save the run in and resume it from, made when missing, used when steps is not 0.	None
keep	str	"last" to keep only the newest checkpoint file, or "all" to keep every one.	'last'
every	int	The number of iterations between two checkpoint files.	1
on_conflict	str	"error" to refuse a checkpoint saved with other settings, or "restart" to discard it.	'error'
callbacks	Any	Functions, or objects with on_step, called with a record of each iteration.	()
chunk	Any	The number of replicates evaluated together, "auto", a positive integer, or None for the lowered study as it is.	'auto'
shard	Any	A pair (i, n) that runs only the replicates whose number modulo n is i, or None for every replicate.	None

Returns:

Type	Description
BootstrapResult	A BootstrapResult.

Raises:

Type	Description
TypeError	If problem is not a lowered study or fit is not an OptResult.
ValueError	If the fit is of another study, did not converge under require_converged=True, or the top level has fewer than three members.

diff_soup.inference.BootstrapResult dataclass

The replicates of a bootstrap run, with their intervals and bands.

boot[handle] is boot.values(handle). Save a result with save, read it back with BootstrapResult.load, and combine the parts of a sharded run with BootstrapResult.merge. A part alone answers only values and laplace.

Attributes:

Name	Type	Description
weights	ndarray	The weights each replicate gave the members, one row per replicate.
members	tuple	The labels of the reweighted members, in the order of the columns of weights.
over	str	The name of the reweighted level.
weight_scheme	str	The weights= of the run, "bayesian" or "multinomial".
weight_priors	bool	The weight_priors= of the run.
steps	Any	The steps= of the run, an integer or "refit".
max_attempts	Any	The step attempts allowed per replicate, or None for steps=0.
outcomes	Any	How each replicate and each leave-one-member-out estimate ended, or None for steps=0.
fit_params	ndarray	The fit's point, where every replicate starts.
fit_status	str	The status the bootstrapped fit ended with.
fit_converged	bool	Whether the bootstrapped fit converged.
shard	Any	The (i, n) of a sharded run's part, or None for a whole run.

n_kept property

n_kept: int

The number of replicates kept for the intervals.

n_failed property

n_failed: int

The number of replicates that failed, left out of the intervals.

values

values(handle: Any, *, members: bool = False) -> Any

Return a parameter's values at the kept replicates, one row per replicate.

A row holds one value per member for a parameter of each member, and a function parameter gives functions in place of arrays. Under weights="multinomial" a member's value is NaN, or None for a function, in the replicates that did not draw it.

Parameters:

Name	Type	Description	Default
handle	Any	A parameter handle of the bootstrapped study.	required
members	bool	Must be True for a parameter of each member.	False

Returns:

Type	Description
Any	The values, stacked over the kept replicates.

interval

interval(statistic: Any, *, level: float = 0.95, kind: str = 'bca', members: bool = False) -> BootstrapInterval

Return the bootstrap interval of a parameter, or of a statistic of the parameters.

The acceleration of kind="bca" comes from the leave-one-member-out estimates. A member's own parameter needs members=True, weights="bayesian" and kind="percentile".

Parameters:

Name	Type	Description	Default
statistic	Any	A parameter handle, or a function statistic(view) where view[handle] is that parameter's value at one estimate.	required
level	float	The coverage, strictly between 0 and 1.	0.95
kind	str	"bca" for the bias-corrected and accelerated interval, or "percentile".	'bca'
members	bool	Whether statistic reads a parameter of each member, as a dictionary from member label to value.	False

Returns:

Type	Description
BootstrapInterval	A BootstrapInterval.

band

band(statistic: Any, *, at: Any, level: float = 0.95, kind: str = 'bca', members: bool = False) -> BootstrapBand

Return the pointwise bootstrap band of a function parameter, or of a statistic, on a grid.

Parameters:

Name	Type	Description	Default
statistic	Any	A function parameter handle, or a function statistic(view, at) returning values on the grid.	required
at	Any	The grid, a non-empty array of finite numbers.	required
level	float	The coverage, strictly between 0 and 1.	0.95
kind	str	"bca" for the bias-corrected and accelerated band, or "percentile".	'bca'
members	bool	Whether statistic reads a parameter of each member, as a dictionary from member label to value.	False

Returns:

Type	Description
BootstrapBand	A BootstrapBand.

laplace

laplace(handle: Any) -> np.ndarray

Return a parameter's Laplace covariance at the fit.

Parameters:

Name	Type	Description	Default
handle	Any	A parameter handle of the bootstrapped study.	required

Returns:

Type	Description
ndarray	The covariance block, or one block per member, stacked, for a parameter of each member.

diff_soup.inference.BootstrapInterval dataclass

A bootstrap interval, returned by BootstrapResult.interval.

The ends are NaN where status is not "ok" or "unresolved", and reason says why.

Attributes:

Name	Type	Description
lower	Any	The lower end, a number or an array shaped like the statistic.
upper	Any	The upper end, shaped like lower.
level	float	The coverage.
kind	str	"bca" or "percentile".
status	Any	"ok", "unresolved", "too_many_failed", "jackknife_failed" or "degenerate", shaped like lower.
reason	str	Sentences explaining any status other than "ok", and any caveat, or an empty string.
n_kept	int	The number of kept replicates.
n_failed	int	The number of failed replicates.
members	int	The number of reweighted members.
centre	Any	The statistic at the fit's point.

diff_soup.inference.BootstrapBand dataclass

A pointwise bootstrap band on a grid, returned by BootstrapResult.band.

It has every attribute of BootstrapInterval, with lower, upper, status and centre given per grid point.

Attributes:

Name	Type	Description
at	ndarray	The grid.
draws	ndarray	The statistic on the grid at each kept replicate, one row per replicate.

---

Prediction

diff_soup.inference.predict

predict(problem, fit_or_draws, *, experiment_id=None, kind='latent', level=0.95)

Carry the parameter uncertainty of a fit or of posterior draws through to the predicted measurements.

From a fit, each prediction is linearized at the fit and the Laplace covariance is propagated. From Draws, the band is the mean and spread of the predictions over the draws.

Parameters:

Name	Type	Description	Default
problem		The InferenceProblem of the fit or the draws.	required
fit_or_draws		A fit as laplace takes it, or the draws of ds.inference.sample on problem.	required
experiment_id		The experiment to predict, required when the problem has several.	None
kind		"latent" for the spread from the parameters alone, or "predictive" with the measurement noise added.	'latent'
level		The coverage of the band, strictly between 0 and 1.	0.95

Returns:

Type	Description
	A Prediction mapping each measurement name to its PredictionBand.

diff_soup.inference.Prediction dataclass

The bands of predict, a mapping from measurement name to PredictionBand.

prediction["u"] is the band of the measurement "u", and prediction.names lists the measurement names.

Attributes:

Name	Type	Description
bands	Mapping	The bands, by measurement name.

diff_soup.inference.PredictionBand dataclass

The prediction band of one measurement over its observation times.

Attributes:

Name	Type	Description
times	Any	The observation times, without the masked-out ones.
mean	Any	The predicted mean at each time.
sd	Any	The predicted standard deviation at each time.
lower	Any	The lower end, mean - z * sd, with z the normal quantile for the level.
upper	Any	The upper end, mean + z * sd.