Likelihoods

AbstractLikelihood

AbstractLikelihood(
    num_datapoints: int,
    integrator: AbstractIntegrator = GHQuadratureIntegrator(),
)

Bases: Module

Abstract base class for likelihoods.

All likelihoods must inherit from this class and implement the predict and link_function methods.

Parameters:

• num_datapoints (int) –

  the number of data points.

• integrator (AbstractIntegrator, default: GHQuadratureIntegrator() ) –

  The integrator to be used for computing expected log likelihoods. Must be an instance of AbstractIntegrator.

__call__

__call__(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  The predictive distribution.

predict abstractmethod

predict(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  npd.Distribution: The predictive distribution.

link_function abstractmethod

link_function(f: Float[Array, ...]) -> npd.Distribution

Return the link function of the likelihood function.

Parameters:

• f (Float[Array, '...']) –

  the latent Gaussian process values.

Returns:

• Distribution –

  npd.Distribution: The distribution of observations, y, given values of the Gaussian process, f.

expected_log_likelihood

expected_log_likelihood(
    y: Float[Array, "N D"],
    mean: Float[Array, "N D"],
    variance: Float[Array, "N D"],
    mean_g: Optional[Float[Array, "N D"]] = None,
    variance_g: Optional[Float[Array, "N D"]] = None,
    **_: Any,
) -> Float[Array, " N"]

Compute the expected log likelihood.

For a variational distribution $q(f)\sim\mathcal{N}(m, s)$ and a likelihood $p(y|f)$, compute the expected log likelihood:

$$\mathbb{E}_{q(f)}\left[\log p(y|f)\right]$$

Parameters:

• y (Float[Array, 'N D']) –

  The observed response variable.

• mean (Float[Array, 'N D']) –

  The variational mean.

• variance (Float[Array, 'N D']) –

  The variational variance.

• mean_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• variance_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• **_ (Any, default: {} ) –

  Unused extra arguments for compatibility with specialised likelihoods.

Returns:

• ScalarFloat ( Float[Array, ' N'] ) –

  The expected log likelihood.

AbstractNoiseTransform

Bases: Module

Abstract base class for noise transformations.

__call__ abstractmethod

__call__(x: Float[Array, ...]) -> Float[Array, ...]

Transform the input noise signal.

moments abstractmethod

moments(
    mean: Float[Array, ...], variance: Float[Array, ...]
) -> NoiseMoments

Compute the moments of the transformed noise signal.

LogNormalTransform

Bases: AbstractNoiseTransform

Log-normal noise transformation.

SoftplusTransform

SoftplusTransform(num_points: int = 20)

Bases: AbstractNoiseTransform

Softplus noise transformation.

AbstractHeteroscedasticLikelihood

AbstractHeteroscedasticLikelihood(
    num_datapoints: int,
    noise_prior: Prior,
    noise_transform: Union[
        AbstractNoiseTransform,
        Callable[[Float[Array, ...]], Float[Array, ...]],
    ] = SoftplusTransform(),
    integrator: AbstractIntegrator = GHQuadratureIntegrator(),
)

Bases: AbstractLikelihood

Base class for heteroscedastic likelihoods with latent noise processes.

supports_tight_bound

supports_tight_bound() -> bool

Return whether the tighter bound from Lázaro-Gredilla & Titsias (2011) is applicable.

noise_statistics

noise_statistics(
    mean: Float[Array, "N D"], variance: Float[Array, "N D"]
) -> NoiseMoments

Moment matching of the transformed noise process.

Parameters:

• mean (Float[Array, 'N D']) –

  Mean of the latent noise GP.

• variance (Float[Array, 'N D']) –

  Variance of the latent noise GP.

Returns:

• NoiseMoments ( NoiseMoments ) –

  Expected log variance, inverse variance, and variance.

predict abstractmethod

predict(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  npd.Distribution: The predictive distribution.

link_function abstractmethod

link_function(f: Float[Array, ...]) -> npd.Distribution

Return the link function of the likelihood function.

Parameters:

• f (Float[Array, '...']) –

  the latent Gaussian process values.

Returns:

• Distribution –

  npd.Distribution: The distribution of observations, y, given values of the Gaussian process, f.

Gaussian

Gaussian(
    num_datapoints: int,
    obs_stddev: Union[
        ScalarFloat, Float[Array, "#N"], NonNegativeReal
    ] = 1.0,
    integrator: AbstractIntegrator = AnalyticalGaussianIntegrator(),
)

Bases: AbstractLikelihood

Gaussian likelihood object.

Parameters:

• num_datapoints (int) –

  the number of data points.

• obs_stddev (Union[ScalarFloat, Float[Array, '#N']], default: 1.0 ) –

  the standard deviation of the Gaussian observation noise.

• integrator (AbstractIntegrator, default: AnalyticalGaussianIntegrator() ) –

  The integrator to be used for computing expected log likelihoods. Must be an instance of AbstractIntegrator. For the Gaussian likelihood, this defaults to the AnalyticalGaussianIntegrator, as the expected log likelihood can be computed analytically.

link_function

link_function(f: Float[Array, ...]) -> npd.Normal

The link function of the Gaussian likelihood.

Parameters:

• f (Float[Array, '...']) –

  Function values.

Returns:

• Normal –

  npd.Normal: The likelihood function.

predict

predict(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.MultivariateNormal

Evaluate the Gaussian likelihood.

Evaluate the Gaussian likelihood function at a given predictive distribution. Computationally, this is equivalent to summing the observation noise term to the diagonal elements of the predictive distribution's covariance matrix.

Parameters:

• dist (Distribution) –

  The Gaussian process posterior, evaluated at a finite set of test points.

Returns:

• MultivariateNormal –

  npd.Distribution: The predictive distribution.

noise_vector

noise_vector(n: int) -> Float[Array, ' N']

Per-observation noise variance vector (scalar broadcast for single-output).

prepare_targets

prepare_targets(
    y: Float[Array, "N 1"], mx: Float[Array, "N 1"]
) -> tuple[Float[Array, "N 1"], Float[Array, "N 1"]]

Return targets and mean in the format expected by the unified predict/MLL path.

__call__

__call__(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  The predictive distribution.

expected_log_likelihood

expected_log_likelihood(
    y: Float[Array, "N D"],
    mean: Float[Array, "N D"],
    variance: Float[Array, "N D"],
    mean_g: Optional[Float[Array, "N D"]] = None,
    variance_g: Optional[Float[Array, "N D"]] = None,
    **_: Any,
) -> Float[Array, " N"]

Compute the expected log likelihood.

For a variational distribution $q(f)\sim\mathcal{N}(m, s)$ and a likelihood $p(y|f)$, compute the expected log likelihood:

$$\mathbb{E}_{q(f)}\left[\log p(y|f)\right]$$

Parameters:

• y (Float[Array, 'N D']) –

  The observed response variable.

• mean (Float[Array, 'N D']) –

  The variational mean.

• variance (Float[Array, 'N D']) –

  The variational variance.

• mean_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• variance_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• **_ (Any, default: {} ) –

  Unused extra arguments for compatibility with specialised likelihoods.

Returns:

• ScalarFloat ( Float[Array, ' N'] ) –

  The expected log likelihood.

MultiOutputGaussian

MultiOutputGaussian(
    num_datapoints: int,
    num_outputs: int,
    obs_stddev: Union[float, Float[Array, " P"]] = 1.0,
)

Bases: Gaussian

Gaussian likelihood with per-output noise variance.

Parameters:

• num_datapoints (int) –

  Total number of observations (N, not N*P).

• num_outputs (int) –

  Number of output dimensions (P).

• obs_stddev (Union[float, Float[Array, ' P']], default: 1.0 ) –

  Per-output noise standard deviation. Scalar broadcasts to [P].

noise_vector

noise_vector(n: int) -> Float[Array, ' NP']

Per-observation noise variance in output-major (Kronecker) order.

Returns sigma_p^2 with each output's variance repeated N times, concatenated across outputs: [σ₁²...σ₁², σ₂²...σ₂², ...].

prepare_targets

prepare_targets(
    y: Float[Array, "N P"], mx: Float[Array, "N 1"]
) -> tuple[Float[Array, "NP 1"], Float[Array, "NP 1"]]

Reshape multi-output targets to output-major long format.

__call__

__call__(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  The predictive distribution.

predict

predict(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.MultivariateNormal

Evaluate the Gaussian likelihood.

Evaluate the Gaussian likelihood function at a given predictive distribution. Computationally, this is equivalent to summing the observation noise term to the diagonal elements of the predictive distribution's covariance matrix.

Parameters:

• dist (Distribution) –

  The Gaussian process posterior, evaluated at a finite set of test points.

Returns:

• MultivariateNormal –

  npd.Distribution: The predictive distribution.

link_function

link_function(f: Float[Array, ...]) -> npd.Normal

The link function of the Gaussian likelihood.

Parameters:

• f (Float[Array, '...']) –

  Function values.

Returns:

• Normal –

  npd.Normal: The likelihood function.

expected_log_likelihood

expected_log_likelihood(
    y: Float[Array, "N D"],
    mean: Float[Array, "N D"],
    variance: Float[Array, "N D"],
    mean_g: Optional[Float[Array, "N D"]] = None,
    variance_g: Optional[Float[Array, "N D"]] = None,
    **_: Any,
) -> Float[Array, " N"]

Compute the expected log likelihood.

For a variational distribution $q(f)\sim\mathcal{N}(m, s)$ and a likelihood $p(y|f)$, compute the expected log likelihood:

$$\mathbb{E}_{q(f)}\left[\log p(y|f)\right]$$

Parameters:

• y (Float[Array, 'N D']) –

  The observed response variable.

• mean (Float[Array, 'N D']) –

  The variational mean.

• variance (Float[Array, 'N D']) –

  The variational variance.

• mean_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• variance_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• **_ (Any, default: {} ) –

  Unused extra arguments for compatibility with specialised likelihoods.

Returns:

• ScalarFloat ( Float[Array, ' N'] ) –

  The expected log likelihood.

Bernoulli

Bernoulli(
    num_datapoints: int,
    integrator: AbstractIntegrator = GHQuadratureIntegrator(),
)

Bases: AbstractLikelihood

Parameters:

• num_datapoints (int) –

  the number of data points.

• integrator (AbstractIntegrator, default: GHQuadratureIntegrator() ) –

  The integrator to be used for computing expected log likelihoods. Must be an instance of AbstractIntegrator.

link_function

link_function(f: Float[Array, ...]) -> npd.BernoulliProbs

The probit link function of the Bernoulli likelihood.

Parameters:

• f (Float[Array, '...']) –

  Function values.

Returns:

• BernoulliProbs –

  npd.Bernoulli: The likelihood function.

predict

predict(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.BernoulliProbs

Evaluate the pointwise predictive distribution.

Evaluate the pointwise predictive distribution, given a Gaussian process posterior and likelihood parameters.

Parameters:

• dist ([npd.MultivariateNormal, GaussianDistribution].) –

  The Gaussian process posterior, evaluated at a finite set of test points.

Returns:

• BernoulliProbs –

  npd.Bernoulli: The pointwise predictive distribution.

__call__

__call__(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  The predictive distribution.

expected_log_likelihood

expected_log_likelihood(
    y: Float[Array, "N D"],
    mean: Float[Array, "N D"],
    variance: Float[Array, "N D"],
    mean_g: Optional[Float[Array, "N D"]] = None,
    variance_g: Optional[Float[Array, "N D"]] = None,
    **_: Any,
) -> Float[Array, " N"]

Compute the expected log likelihood.

For a variational distribution $q(f)\sim\mathcal{N}(m, s)$ and a likelihood $p(y|f)$, compute the expected log likelihood:

$$\mathbb{E}_{q(f)}\left[\log p(y|f)\right]$$

Parameters:

• y (Float[Array, 'N D']) –

  The observed response variable.

• mean (Float[Array, 'N D']) –

  The variational mean.

• variance (Float[Array, 'N D']) –

  The variational variance.

• mean_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• variance_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• **_ (Any, default: {} ) –

  Unused extra arguments for compatibility with specialised likelihoods.

Returns:

• ScalarFloat ( Float[Array, ' N'] ) –

  The expected log likelihood.

Poisson

Poisson(
    num_datapoints: int,
    integrator: AbstractIntegrator = GHQuadratureIntegrator(),
)

Bases: AbstractLikelihood

Parameters:

• num_datapoints (int) –

  the number of data points.

• integrator (AbstractIntegrator, default: GHQuadratureIntegrator() ) –

  The integrator to be used for computing expected log likelihoods. Must be an instance of AbstractIntegrator.

link_function

link_function(f: Float[Array, ...]) -> npd.Poisson

The link function of the Poisson likelihood.

Parameters:

• f (Float[Array, '...']) –

  Function values.

Returns:

• Poisson –

  npd.Poisson: The likelihood function.

predict

predict(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Poisson

Evaluate the pointwise predictive distribution.

Evaluate the pointwise predictive distribution, given a Gaussian process posterior and likelihood parameters.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The Gaussian process posterior, evaluated at a finite set of test points.

Returns:

• Poisson –

  npd.Poisson: The pointwise predictive distribution.

__call__

__call__(
    dist: Union[MultivariateNormal, GaussianDistribution],
) -> npd.Distribution

Evaluate the likelihood function at a given predictive distribution.

Parameters:

• dist (Union[MultivariateNormal, GaussianDistribution]) –

  The predictive distribution to evaluate the likelihood at.

Returns:

• Distribution –

  The predictive distribution.

expected_log_likelihood

expected_log_likelihood(
    y: Float[Array, "N D"],
    mean: Float[Array, "N D"],
    variance: Float[Array, "N D"],
    mean_g: Optional[Float[Array, "N D"]] = None,
    variance_g: Optional[Float[Array, "N D"]] = None,
    **_: Any,
) -> Float[Array, " N"]

Compute the expected log likelihood.

For a variational distribution $q(f)\sim\mathcal{N}(m, s)$ and a likelihood $p(y|f)$, compute the expected log likelihood:

$$\mathbb{E}_{q(f)}\left[\log p(y|f)\right]$$

Parameters:

• y (Float[Array, 'N D']) –

  The observed response variable.

• mean (Float[Array, 'N D']) –

  The variational mean.

• variance (Float[Array, 'N D']) –

  The variational variance.

• mean_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• variance_g (Float[Array, 'N D'], default: None ) –

  Optional moments of the latent noise process for heteroscedastic likelihoods.

• **_ (Any, default: {} ) –

  Unused extra arguments for compatibility with specialised likelihoods.

Returns:

• ScalarFloat ( Float[Array, ' N'] ) –

  The expected log likelihood.

inv_probit

inv_probit(x: Float[Array, ' *N']) -> Float[Array, ' *N']

Compute the inverse probit function.

Parameters:

• x (Float[Array, '*N']) –

  A vector of values.

Returns

Float[Array, "*N"] The inverse probit of the input vector.