Brief: Entropy Potential for Volumetric Recurrent Computing

Objective

Define and implement quantitative measures of the information capacity and dynamical richness of a candidate VRC substrate before evaluating it on tasks such as MNIST.

For VRC, “entropy potential” does not mean thermodynamic entropy. It means the number and structure of physically distinguishable computational states and trajectories available to the system.

The initial implementation should distinguish three quantities rather than collapse them into one scalar.

1. Configuration Entropy

Measure how much learned structure the physical substrate can embody.

Let the substrate contain (N_\Theta) independently configurable physical degrees of freedom, with node (i) supporting (K_i) reliably distinguishable states.

The raw configuration entropy is

[ H_\Theta^{\max}

\sum_i \log_2 K_i. ]

For uniform (K),

[ H_\Theta^{\max}

N_\Theta \log_2 K. ]

Example:

[ N_\Theta=10^9,\qquad K=4 ]

gives

[ H_\Theta^{\max}=2\times10^9\ \text{bits}. ]

This is only an upper bound.

Real configuration capacity must account for:

  • programming errors;
  • correlated node states;
  • fabrication constraints;
  • crosstalk;
  • drift;
  • state ambiguity.

A more meaningful eventual physical metric is

[ I( \Theta_{\text{commanded}}; \Theta_{\text{realized}} ), ]

the mutual information between requested and reproducibly realized configurations.

Call this quantity:

[ \boxed{H_\Theta} ]

or model/configuration entropy.

2. Transient-State Capacity

Measure how much distinguishable information can exist in the transient state (S_t).

For photonic VRC, model a propagation operator

[ H:S_{\text{in}}\rightarrow S_{\text{out}}. ]

Compute or estimate its singular-value decomposition:

[ H=U\Sigma V^\dagger. ]

The singular values

[ \sigma_1,\sigma_2,\ldots ]

identify independent optical channels supported by the substrate.

Noise determines which channels are actually usable.

A first capacity estimate is

[ C_S

\sum_i \log_2 \left( 1+\mathrm{SNR}_i \right). ]

The useful quantity is therefore not simply pixel count or voxel count.

It is:

[ \boxed{ \text{number and quality of physically distinguishable optical modes} } ]

supported under the actual:

  • wavelength;
  • optical bandwidth;
  • aperture;
  • polarization;
  • power;
  • loss;
  • detector noise;
  • material noise;
  • coupling constraints.

Call this:

[ \boxed{C_S} ]

or transient-state capacity.

3. Raw Volumetric Modal Estimate

Before detailed simulation, provide a rough physical upper bound.

For a dielectric volume (V), refractive index (n), vacuum wavelength (\lambda), and fractional optical bandwidth

[ \beta=\frac{\Delta f}{f}, ]

estimate the electromagnetic mode count as

[ N_{\text{modes}} \sim 8\pi n^3 \frac{V}{\lambda^3} \beta. ]

This is a sanity-check quantity only.

It estimates the raw modal richness of the volume, not the number of independently controllable or computationally useful variables.

Report it separately from (C_S).

4. Dynamical Entropy

VRC is recurrent:

[ S_{t+1}=F_\Theta(S_t,Q). ]

Therefore raw state-space size is insufficient.

We must determine whether recurrence:

  • preserves distinctions;
  • destroys distinctions;
  • creates useful expansion;
  • collapses into attractors;
  • becomes chaotic;
  • supports many distinct trajectories.

Linearize the recurrent operator around a trajectory:

[ J_t

\frac{\partial F_\Theta}{\partial S} \Bigg|_{S_t}. ]

For (T) recurrences, study

[ J_{T-1} J_{T-2} \cdots J_0. ]

Estimate its:

  • leading singular values;
  • effective rank;
  • Lyapunov spectrum;
  • contraction/expansion rates;
  • trajectory divergence under small input perturbations.

Define a provisional quantity

[ \boxed{H_{\text{dyn}}(T)} ]

representing recurrent trajectory richness.

Do not require a final theoretical definition in the first implementation. Start with empirical metrics.

Important Distinction

A substrate can have enormous raw modal capacity while being computationally trivial.

For example:

[ N_{\text{modes}}\gg1 ]

but

[ S_t\rightarrow \text{one of a few attractors} ]

after one or two recurrences.

Conversely, a smaller state space may support rich controllable trajectories.

Therefore VRC entropy potential should initially be represented as a vector:

[ \boxed{ \mathcal E_{\text{VRC}}

\left( H_\Theta, C_S, H_{\text{dyn}}(T) \right) } ]

where:

  • (H_\Theta): model/configuration entropy;
  • (C_S): transient-state information capacity;
  • (H_{\text{dyn}}(T)): recurrent dynamical richness.

Do not prematurely combine these into one score.

Implementation Goal

Add an entropy-characterization subsystem to cintamani.

Suggested command:

cargo run --release -- entropy configs/<substrate>.toml

It should characterize a candidate VRC operator independently of any downstream ML task.

Initial Software Measurements

For a simulated operator (F_\Theta), report:

  • total configurable parameter count;
  • quantization/state count per parameter;
  • raw (H_\Theta^{\max});
  • estimated effective configuration entropy;
  • transient-state dimension;
  • estimated leading singular spectrum;
  • effective rank;
  • condition number where meaningful;
  • state capacity proxy;
  • Jacobian singular spectrum;
  • state contraction/expansion by recurrence;
  • perturbation survival after (T) recurrences;
  • trajectory divergence;
  • apparent attractor count;
  • limit-cycle detection;
  • state norm/intensity evolution.

Evaluate over

[ T\in{1,2,4,8,16,32}. ]

Matrix-Free Requirement

Do not explicitly construct a full transmission matrix or Jacobian when state dimensions become large.

Use:

  • Jacobian-vector products;
  • vector-Jacobian products;
  • randomized SVD;
  • power iteration;
  • Lanczos-style methods where useful;
  • sampled perturbation probes.

The entropy subsystem must remain usable when the state operator becomes too large to materialize.

Experimental Physical Mapping

Design the software API so the same measurements can later be estimated from a real physical substrate.

For a real slab:

  1. inject known optical perturbations;
  2. measure resulting output fields;
  3. reconstruct or approximate the input-output operator;
  4. estimate its singular spectrum;
  5. perturb recurrent states;
  6. measure divergence/contraction over multiple passes.

The eventual physical substrate should therefore be characterizable without requiring a complete microscopic simulation.

First Falsification Question

Before training MNIST, determine whether the candidate substrate has a useful operating regime satisfying roughly:

[ \boxed{ \text{high accessible state capacity} + \text{nontrivial recurrence}

\text{uncontrolled chaos}

\text{rapid collapse} } ]

A candidate operator that immediately destroys almost all distinguishable state should not advance to ML testing.

Likewise, uncontrolled exponential divergence should be treated as failure unless training can reliably exploit it.

Deliverables

Implement:

src/
  entropy/
    configuration.rs
    modes.rs
    singular.rs
    dynamics.rs
    perturbation.rs
    report.rs

Produce:

output/
  entropy-summary.json
  singular-values.csv
  recurrence-dynamics.csv

and a human-readable report containing:

  1. Configuration Capacity
  2. Transient-State Capacity
  3. Recurrent Dynamics
  4. Observed Failure Modes
  5. Recommendation: advance / modify / reject

Governing Question

The entropy work exists to answer:

How much controllable computational possibility does this physical volume actually contain?

Only after that question has a plausible answer should VRC performance on tasks such as recursive MNIST be interpreted seriously.