Out-of-Distribution Generalization in Sequential Data Research on generalization in sequential and structured data highlights significant challenges regarding distribution shifts and model capacity. Time series frequently manifest distribution shifts and non-stationary learning dynamics that pose challenges for out-of-distribution generalization ^[9]. Generalization bounds for autoregressive processes relate empirical loss to denoising loss, characterizing sample complexity in terms of tokens rather than sequences ^[10]. However, theoretical analysis of generalization in multi-label learning remains in the early stage, with open problems regarding label correlations and inductive biases ^[7]. When training data is not independent and identically distributed, algorithm performances are challenged by catastrophic forgetting, and regularization-based approaches may fail to discriminate classes from different tasks ^[17]. These findings collectively establish that standard assumptions often break down in sequential settings, necessitating careful evaluation protocols, though specific claims about unseen users versus random splits require empirical verification beyond survey-level summaries. Entropy and Weight Structure in Regularization Parallel work examines the role of entropy and weight structure in regularization and robustness. Prior work attempted to test whether explicitly maximizing the entropy of multiplicative weight-noise parameters during training improves held-out classification accuracy on subject-disjoint splits ^[1]. That study carries a retraction notice because the executed intervention did not implement the preregistered one, so its reported effect cannot be attributed to entropy maximization; it may be cited only as prior work that attempted the question and whose result is not interpretable. Beyond this, the relationship between the entropy of intermediate representations and a model's robustness to distributional shift has been studied, showing that removal of low-entropy bits can benefit out-of-distribution performance ^[5]. In Bayesian contexts, weight matrices often exhibit fast singular value decay, suggesting low-rank parameterization may be sufficient without standard parameter costs ^[14]. Regularization characterizes optimal learning in settings where empirical risk minimization fails, with maximum entropy programs exhibiting optimal learners in multiclass settings ^[6]. Sharpness-Aware Minimization algorithms improve neural network generalization by steering model parameters away from sharp regions, though the underlying mechanisms are not fully understood ^[2]. Together, this literature suggests entropy and spectral properties are relevant to robustness, but the direction of effect (minimization versus maximization) and the target (weights versus representations) remain distinct questions. Projections and Dimensionality Reduction Methods involving projections offer alternative perspectives on dimensionality and regularization. Reducing linear program sizes using random and data-driven projections can accelerate solving while maintaining solution quality ^[3]. In image processing, denoising engines serve as regularization strategies for inverse problems through fixed-point projection ^[12]. These approaches demonstrate that projection-based constraints can enforce structure, yet their application to spectral entropy minimization in temporal input layers is novel. While text watermarking presents challenges in low-entropy texts, this domain differs significantly from weight spectrum regularization in classifiers ^[16]. The existing projection literature supports the feasibility of learned constraints but does not settle whether spectral entropy minimization aids temporal feature learning. This work positions itself against these themes by testing a specific spectral intervention on input projections. Unlike prior attempts to maximize weight noise entropy ^[1], we minimize spectral entropy on deterministic weights. Unlike low-rank Bayesian methods ^[14], we operate in a standard deterministic training regime with a specific entropy term. We address the gap where no investigation exists into whether minimizing the spectral entropy of input projections specifically improves subject-disjoint generalization in time-series transformers.

The experiment tests the hypothesis that enforcing a low-entropy singular value spectrum on input projection weights enhances feature learning for hard temporal tasks compared to standard weight decay. The intervention is predicted to yield higher held-out classification accuracy than the comparator, with mean paired contrast in the hypothesised direction exceeding the fixed worthwhile margin of 0.0125098 proportion. This prediction rests on the assumption that concentrating energy into dominant temporal modes via spectral entropy minimization facilitates more robust feature extraction than distributing energy uniformly, which standard weight decay permits. The core operator is a regularization term added to the training objective that enforces low-entropy singular value spectra on the input projection weights. During each optimization step, the operator computes the singular value decomposition (SVD) of the input projection weight matrix. The resulting singular values are normalized to form a probability distribution. The operator then calculates the Shannon entropy of this distribution and adds it to the loss function with a positive coefficient, such that minimizing the total loss requires minimizing the spectral entropy. This encourages the model to select dominant temporal modes rather than distributing energy uniformly across all singular vectors. The comparator neutralizes this specific spectral pressure by applying standard weight decay without the spectral entropy term, isolating the effect of the spectral constraint from general regularization. The hypothesis targets the mechanistic role of weight spectra in generalization. If spectral entropy minimization improves accuracy, it suggests that finite-width networks benefit from input layers that act as selective filters rather than broad projectors. Conversely, if the intervention yields no gain, it implies that standard weight decay already suffices to manage spectral properties or that spectral entropy is not the critical factor for subject-disjoint robustness in this architecture. The preregistered direction was refuted if the mean contrast fails to exceed the margin or if the confidence interval includes zero. We report both primary and confirmatory-replication cohorts separately to distinguish initial findings from replicated evidence. The method avoids conflating spectral entropy with weight magnitude. Standard weight decay constrains the norm of weights but does not constrain the distribution of singular values; a matrix can have small norm but high spectral entropy (uniform singular values) or low spectral entropy (one dominant singular value). By comparing the proposed condition against a weight decay baseline, we isolate the spectral distribution effect. The negative controls further rule out confounding factors such as optimization instability or arbitrary loss term addition. This design ensures that any observed effect can be attributed to the spectral entropy minimization rather than auxiliary training dynamics.

Operator specification. SpectralEntropyRegularization is applied to input_proj.weight. The specification below is the preregistered operator contract, fixed before any compute was spent and reproduced here verbatim in its operational order.

Algorithm 1 gives the operator in full.

Control arm. Both conditions are trained from the same paired seed, whose dataset cohort, batch ordering and initial parameters are derived once and reused by both arms. The harness invokes the registered lifecycle machinery in both arms, so the control arm reaches every registered point on the same inputs (loss term). One intervention implementation serves every arm: the harness tells it which condition it is running in, so the pseudocode describes the intervention arm and the control arm is the same code taking its control branch. If the condition the harness passes in is a control condition, return here: add no term to the training objective, and leave the registered target parameter and that parameter's optimizer momentum buffer exactly as received. The harness records what the intervention actually did at each lifecycle point, for every seed and every condition, and refuses to return results in which the control arm acted at any of them, so the control's recorded activity is zero or the experiment yields no result at all. Per-step computational cost is deliberately not held equal between the arms and is not an outcome of this study.

The study design employs a subject-disjoint split to test robustness to unseen users, as standard random splits may hide failure modes specific to distribution shift. Pairing every condition on one seed buys comparability across conditions by controlling for random initialization and data ordering effects, though it costs independence between condition estimates within a seed. We use 43 primary seeds and 43 disjoint confirmatory-replication seeds to ensure sufficient power to detect the preregistered worthwhile margin. The identities of these seeds are recorded in the appendix; we refer to the counts in the prose. This many seeds allows us to construct bootstrap confidence intervals over the paired contrasts, providing a robust estimate of uncertainty beyond single-split evaluations. The first negative control rules out the possibility that any regularization term improves performance simply by acting as a noise source during optimization. The second negative control rules out the possibility that the computational overhead of the SVD operation inadvertently alters training dynamics in a way that benefits accuracy. Without these controls, an improvement in the proposed condition would be indistinguishable from generic regularization effects or implementation artifacts. The analysis computes the mean held-out classification accuracy for each condition and the paired contrast between the proposed condition and the baseline. We report 95% bootstrap confidence intervals over seeds for all metrics. The outcome is defined as the number of correct activity predictions divided by 2,947 windows in the official subject-disjoint test split. These configuration details are fixed across all conditions to ensure that differences in accuracy arise from the regularization term rather than architectural or hyperparameter variations. The design prioritizes internal validity by holding all factors constant except the spectral entropy term. Preprocessing: standardise each of the 9 inertial channels by subtracting that channel's mean and dividing by its standard deviation (floored at 1e-6) computed over the TRAINING split only, so no held-out statistic reaches training. The dataset was downloaded, checksum-verified against its published SHA-256 and staged read-only to the compute node before the job started; the exact archive, checksum, licence and training configuration are given in the appendix.

We report results for the primary and confirmatory-replication cohorts separately, stating each preregistered decision. In the primary cohort, across 4 conditions and 43 seeds, the proposed condition achieved a mean held-out classification accuracy of 0.9053 (95% CI [0.902, 0.908]), while the baseline achieved 0.9049 (95% CI [0.901, 0.908]). The negative controls performed comparably to the baseline at 0.9049 (95% CI [0.901, 0.908]). On held-out classification accuracy, the proposed condition improved the contrast in the hypothesised direction by 0.0004419 versus the baseline condition (95% CI [-0.00213, 0.0031], paired randomisation p=0.75, Cohen's dz=0.05). This difference is not statistically significant as the 95% CI includes zero. The preregistered decision was inconclusive because the contrast did not exceed the fixed worthwhile margin of 0.0125098 proportion. Figure 1 displays the final metric values per condition with 95% bootstrap confidence intervals over seeds for the primary cohort, illustrating the overlap between the proposed condition and the baseline. Figure 2 shows the mean held-out classification accuracy over training steps for each condition, indicating that learning curves tracked closely throughout training without divergence. In the confirmatory replication cohort, across 4 conditions and 43 seeds, the proposed condition achieved a mean held-out classification accuracy of 0.9087 (95% CI [0.906, 0.912]), while the baseline achieved 0.9061 (95% CI [0.902, 0.91]). The negative controls matched the baseline at 0.9061 (95% CI [0.902, 0.91]). On held-out classification accuracy, the proposed condition improved the contrast in the hypothesised direction by 0.002549 versus the baseline condition (95% CI [-0.000331, 0.0055], paired randomisation p=0.106, Cohen's dz=0.25). This difference is also not statistically significant, with the 95% CI including zero. The preregistered decision was inconclusive. Figure 3 displays the final metric values per condition with 95% bootstrap confidence intervals over seeds for the replication cohort, confirming the pattern observed in the primary cohort. Figure 4 shows the mean held-out classification accuracy over training steps for each condition in replication, again showing parallel learning trajectories. The effect size in the replication cohort (dz=0.25) was larger than in the primary cohort (dz=0.05), but the confidence interval still included zero and fell short of the preregistered margin. We do not pool the cohorts or call a primary-only result replicated; both cohorts independently failed to support the hypothesis. The results indicate that spectral entropy minimization did not yield a robust accuracy gain over standard weight decay in this setting. Registered-report decision: the primary cohort was classified as inconclusive and the independent confirmatory replication was classified as inconclusive under the preregistered confidence-interval rule. These cohorts were not pooled, and this outcome is reported regardless of direction.

The quantitative findings indicate that minimizing spectral entropy of input projections does not enhance subject-disjoint generalization accuracy beyond standard weight decay in this transformer architecture. This result contrasts with the hypothesis that concentrating energy into dominant temporal modes would facilitate feature learning. In the context of prior work, this null result clarifies the limits of spectral regularization. While weight matrices often exhibit fast singular value decay ^[14], actively enforcing low-entropy spectra via optimization did not translate to discriminative gains. This suggests that natural spectral decay observed in trained networks ^[14] may be a consequence rather than a cause of generalization, or that weight decay alone suffices to induce beneficial spectral properties without explicit entropy terms. The findings also relate to prior attempts to manipulate weight entropy for robustness. Previous work attempted to test whether maximizing entropy of weight noise improves accuracy but carries a retraction notice rendering its result uninterpretable ^[1]. Our study implements the intervention correctly but finds no benefit to spectral entropy manipulation in the opposite direction (minimization). This suggests that entropy-based regularization of weights, whether maximization or minimization, may not be the critical lever for robustness in this regime. Work on latent variable entropy shows that removal of low-entropy bits can benefit out-of-distribution performance ^[5], but our results suggest this does not extend to input weight spectra. The distinction between representation entropy ^[5] and weight spectral entropy appears meaningful; constraining the input projection did not replicate the benefits observed in latent space masking. Regularization characterizes optimal learning where empirical risk minimization fails ^[6], and sharpness-aware methods steer parameters away from sharp regions ^[2]. However, our results suggest that spectral entropy is not the relevant geometry for this benefit in temporal transformers. The subject-disjoint split ensures evaluation reflects robustness to unseen users ^[9], yet the intervention failed to improve performance under this shift. This aligns with observations that regularization-based approaches can be challenged when data is not i.i.d. ^[17]. The null result across both primary and replication cohorts strengthens the conclusion that spectral entropy minimization is not a reliable method for improving accuracy in this setting. Practitioners should prioritize standard regularization over spectral entropy terms for resource-constrained edge AI where subject-disjoint robustness is critical.

This study is limited to a single dataset (UCI HAR) and architecture (Transformer encoder), which may restrict generalizability to other temporal domains or model classes. The spectral entropy term was applied only to the input projection weights; applying it to deeper attention layers might yield different results. The worthwhile margin was fixed preregistered, but smaller effects might still be practically relevant in specific high-stakes applications. The experiment used deterministic training with a spectral regularizer; combining this with Bayesian methods or ensembles was not explored. Finally, while subject-disjoint splits test robustness to unseen users, other distribution shifts such as sensor drift or activity definition changes were not evaluated. Declared scope. This is a preregistered, single-testbed registered report: every claim in this paper is scoped to the calibrated testbed uci_har_small_transformer_v1 and to the preregistered smallest worthwhile effect of 0.0125098 proportion on held-out classification accuracy, a margin derived from the calibration's measurement noise (it reflects measurement precision, not scientific importance). Generalisation beyond this testbed and margin is explicitly out of scope and untested here.

This study tested whether minimizing spectral entropy of input projection weights improves held-out classification accuracy compared to standard weight decay. The results were inconclusive: the proposed condition did not exceed the preregistered worthwhile margin in either the primary or replication cohort, and confidence intervals included zero. This finding changes practice by indicating that spectral entropy minimization is not a necessary addition to standard weight decay for subject-disjoint robustness in temporal transformers. Future work should investigate whether spectral constraints on deeper layers or in combination with other regularization techniques might yield benefits, or whether spectral properties are merely epiphenomenal to generalization in this regime.