Transformers have triggered great interest in the time series community due to the ability to capture long-range dependencies and interactions ^[6]. Several works propose tackling forecasting problems with Transformers, such as those demanding high prediction capacity to capture precise long-range dependency coupling ^[10]. Others focus on enhancing locality and breaking the memory bottleneck of Transformer on time series forecasting by addressing locality-agnostics and memory costs ^[11]. Efficient designs based on segmentation of time series into subseries-level patches have been proposed to retain local semantic information while reducing computation ^[9]. However, research questions the validity of this line of research despite growing performance, noting that simple linear models can outperform Transformer-based solutions in long-term forecasting tasks ^[4]. Regarding architectural mechanisms, the original Transformer architecture connects the encoder and decoder through an attention mechanism, dispensing with recurrence and convolutions entirely ^[7]. Subsequent work introduces parameter-free changes to the standard attention mechanism which reduces attention to unneeded elements, consistently improving performance in various model sizes ^[8]. Investigations into mechanisms underlying the ability for compositional generalization reformulate multi-head attention as a hypernetwork, revealing latent codes that specify key-query specific operations ^[33]. In terms of position encoding, transformers require effective position encoding to capture the ordering of the time series data, yet the efficacy of absolute versus relative encoding remains controversial ^[5]. Self-supervised representation learning for time series argues that the domain is fundamentally different in nature to either vision or natural language processing with respect to meaningful tasks ^[27]. Large pre-trained time series models observe that tackling time-series analysis tasks usually involves designing and training a separate model, proposing adaptive segmentation to identify optimal dataset-specific strategies ^[18]. However, citation audits require careful alignment between claims and abstract content; for instance, some papers focus on shape analysis for time series by representing data as deformations of a reference, rather than architectural inductive biases broadly ^[19]. Similarly, clustering stability evaluation approaches extend mutual influence ideas using membership degrees, distinct from weight-space regularization comparisons ^[20]. Regularization for temporal feature learning often assumes specific weight geometries facilitate generalization under distribution shift. One study tests whether enforcing a low-entropy singular value spectrum on input projection weights enhances held-out classification accuracy, reporting inconclusive results where neither estimate exceeded the preregistered margin and confidence intervals included zero ^[2]. Another study investigates whether explicitly maximizing the entropy of multiplicative weight noise parameters improves out-of-distribution accuracy, but carries a retracted notice from the journal that published it due to implementation mismatches with the preregistered intervention ^[1]. These findings suggest that spectral entropy minimization of input projections does not yield robust accuracy gains over standard weight decay in this regime. Uncertainty estimates are agreed to be missing from standard networks and Bayesian formalisms are the usual answer, but the same literature reports that the approximations needed to make them tractable are where the guarantees are lost — precisely the regime this study operates in. The present work positions itself against this background by testing a specific cross-layer consistency bias. Unlike weight-space regularization that targets input projections ^[2], our intervention targets attention query projections across depth. Unlike segmentation-based approaches that modify input tokens ^[9], we modify the training objective to encourage stability across layers. This distinction is critical because layer-specific adaptation may be essential for architectures processing inertial signals where subject variability manifests in temporal dynamics rather than just input distribution.

The core operator is a Cross-Layer Query Consistency Loss added to the training objective. This intervention regularizes deeper block query projection weights toward shallower block weights. Specifically, a Frobenius norm consistency regularization term is added during training that penalizes the difference between block 1's query projection weights and block 0's query projection weights (detached). The loss term is computed as 0.01 * ||W_q1 - detach(W_q0)||_F^2, added to the classification objective. Gradients flow only to block 1's weights. This design tests the hypothesis that enforcing stability across depth improves generalization. The comparator is the baseline condition. Both intervention and baseline conditions use identical seeds, data ordering, initialization, and training steps. The only difference is whether the consistency loss term is added. The control arm is defined by what the harness guarantees rather than by a description of the intervention. Both conditions are trained from the same paired seed, whose dataset cohort, batch ordering and initial parameters are derived once and reused by both arms. Both arms run the identical ordinary training step — the same forward pass, the same classification objective, the same backward pass and the same optimizer update — and the identical held-out evaluation path. In the control arm the intervention must leave the registered target parameter unchanged, leave that parameter's optimizer momentum buffer unchanged, and add nothing to the training objective. The harness records what the intervention actually did at each lifecycle point, for every seed and every condition, and refuses to return results in which the control arm acted at any of them. A final LayerNorm(64) precedes mean-pool over time and a Linear(64,6) classification head. This architecture is selected to be large enough to learn temporal representations but small enough to train across many seeds for statistical power. The intervention targets the query projection weights within the self-attention modules, as these determine how temporal tokens attend to one another. The paired contrast in the hypothesised direction H_s is the difference in held-out classification accuracy between the proposed condition and the baseline. The preregistered decision rule compares the mean H_s against a fixed worthwhile margin of 0.0125098 proportion. If the lower bound of the 95% confidence interval exceeds this margin, the hypothesis is supported. If the interval includes zero, the result is inconclusive. This margin was set to reflect a practically significant improvement in activity recognition accuracy. The method isolates the effect of the consistency loss by pairing every condition on one seed, buying control over initialization variance at the cost of computational doubling per seed.

Operator specification. CrossLayerQueryConsistency is applied to blocks.1.attn.q_proj.weight. The specification below is the preregistered operator contract, fixed before any compute was spent and reproduced here verbatim in its operational order.

Algorithm 1 gives the operator in full.

Control arm. Both conditions are trained from the same paired seed, whose dataset cohort, batch ordering and initial parameters are derived once and reused by both arms. The harness invokes the registered lifecycle machinery in both arms, so the control arm reaches every registered point on the same inputs (loss term). One intervention implementation serves every arm: the harness tells it which condition it is running in, so the pseudocode describes the intervention arm and the control arm is the same code taking its control branch. If the condition the harness passes in is a control condition, return here: add no term to the training objective, and leave the registered target parameter and that parameter's optimizer momentum buffer exactly as received. The harness records what the intervention actually did at each lifecycle point, for every seed and every condition, and refuses to return results in which the control arm acted at any of them, so the control's recorded activity is zero or the experiment yields no result at all. Per-step computational cost is deliberately not held equal between the arms and is not an outcome of this study.

A subject-disjoint split tests what this question asks because it evaluates generalization to unseen subjects, whereas a random split would have hidden subject-specific overfitting. This design choice ensures that accuracy gains reflect robustness to inter-individual variability in movement styles and sensor characteristics rather than memorization of subject identities. We employed 43 primary seeds and 43 disjoint confirmatory-replication seeds. The identities are recorded in the appendix. We refer to the counts, never list them. Using this many seeds allows us to estimate the variation attributable to the seeds that produced them, rather than attributing variation to subjects. Each observation is one seed. Pairing every condition on one seed buys control over random initialization and batch ordering variance. The negative control rules out effects from the harness lifecycle machinery itself; without it, changes in evaluation paths would be indistinguishable from intervention effects. The first negative control uses the same harness but without the loss term active, ensuring the baseline is truly passive. Optimization uses SGD(lr=0.05,momentum=0.9). Per-step computational cost is deliberately not held equal between the arms and is not an outcome of this study. The analysis computes bootstrap 95% confidence intervals for mean accuracy and mean difference. We report both primary and confirmatory-replication cohorts separately. We never pool them or call a primary-only result replicated. The pre-specified confirmatory replication on disjoint seeds provides a second estimate of the effect size under identical design conditions but independent randomization. This setup justifies the design by ensuring that any observed difference is due to the consistency loss and not data split variance. The subject-disjoint split is the critical constraint; if we used a random split, the generalization claim would not hold for unseen subjects. The seed pairing ensures that the paired contrast H_s removes seed-specific noise. The negative control confirms that the harness does not introduce bias. The replication cohort confirms whether the primary result was a false positive or stable. Variation across observations is attributed to the seeds that produced them, as each seed generates one paired observation of proposed and baseline accuracy. Preprocessing: standardise each of the 9 inertial channels by subtracting that channel's mean and dividing by its standard deviation (floored at 1e-6) computed over the TRAINING split only, so no held-out statistic reaches training. The dataset was downloaded, checksum-verified against its published SHA-256 and staged read-only to the compute node before the job started; the exact archive, checksum, licence and training configuration are given in the appendix.

We walk through the evidence from the primary and replication cohorts. Figure 1 shows the methodology overview, including data flow, model architecture, the CrossLayerQueryConsistency intervention at the loss term, and subject-disjoint evaluation. In the primary cohort, we report each metric's mean with a bootstrap 95% confidence interval (Table 1). On held-out classification accuracy, the proposed condition mean was 0.906 (95% CI [0.9022, 0.9106]) and the baseline mean was 0.905 (95% CI [0.9016, 0.9086]). The proposed condition improved the contrast in the hypothesised direction by 0.00133 versus the baseline condition (95% CI [-0.00315, 0.0056], paired randomisation p=0.546, Cohen's dz=0.09). Figure 2 displays these means with error bars. The 95% CI includes zero, indicating no statistically significant difference. The preregistered decision was inconclusive. Figure 3 plots the paired mean difference over 43 paired seeds for the primary cohort. Zero is the value the preregistered direction is tested against. The 43 individual paired differences are plotted behind the interval at their exact values. The distribution centers near zero, consistent with the mean difference of 0.00133. Figure 4 shows mean held-out classification accuracy over training steps for the proposed, baseline and negative control 1 conditions in the primary cohort. Each line is the mean over 43 seeds. The curves coincide closely, distinguished by marker shape and dash pattern. This suggests the consistency loss did not alter training dynamics substantially. In the pre-specified confirmatory replication on disjoint seeds, we report results separately (Table 3). On held-out classification accuracy, the proposed condition mean was 0.906 (95% CI [0.902, 0.909]) and the baseline mean was 0.908 (95% CI [0.904, 0.911]). The proposed condition worsened the contrast in the hypothesised direction by -0.00157 versus the baseline condition (95% CI [-0.00487, 0.00182], paired randomisation p=0.372, Cohen's dz=-0.14). Table 4 details this outcome. The 95% CI includes zero, and the direction reversed compared to the primary cohort. The preregistered decision was inconclusive. Figure 5 displays the replication cohort means. Figure 6 plots the paired mean difference for the replication cohort, showing the distribution of the 43 individual paired differences behind the interval. Figure 7 shows accuracy over training steps for the replication cohort, where curves again coincide. Variation across observations is attributed to the seeds that produced them. The primary cohort showed a negligible positive shift, while the replication cohort showed a negligible negative shift. Neither estimate exceeded the preregistered margin of 0.0125098 proportion. The negative control 1 condition achieved 0.908 in the primary cohort and 0.907 in the replication cohort, confirming the harness did not introduce bias. The results do not support the hypothesis that cross-layer consistency improves subject-disjoint generalization in this regime. Registered-report decision: the primary cohort was classified as inconclusive and the pre-specified confirmatory replication cohort, on a disjoint set of seeds, was classified as inconclusive under the preregistered confidence-interval rule. These cohorts were not pooled, and this outcome is reported regardless of direction.

The principal finding is that enforcing cross-layer consistency in attention query projections did not improve held-out classification accuracy on the UCI HAR dataset. The mean difference was 0.00133 in the primary cohort and -0.00157 in the replication cohort, with confidence intervals including zero in both cases. This suggests that transformer depth does not benefit from query weight stability across blocks for this task. The result is consistent with literature reporting that approximations needed to make regularization tractable are where guarantees are lost. It aligns with findings that spectral entropy minimization of input projections does not yield robust accuracy gains over standard weight decay in this regime ^[2]. What this design does establish is that the intervention was implemented correctly, as confirmed by the negative control and harness logs. What it does not establish is that cross-layer consistency is useless for all time-series tasks; the UCI HAR setting involves short windows (128 timesteps) where deep hierarchical processing may not be required. A mechanism that would produce the observed result is that layer-specific adaptation is essential for this architecture class, allowing deeper blocks to refine attention patterns for complex temporal features rather than copying shallow projections. An alternative mechanism is that the regularization strength (0.01 coefficient) was too weak to constrain weights meaningfully, or too strong to allow necessary divergence. The pre-specified confirmatory replication on disjoint seeds adds confidence that the primary result was not a false positive, as the effect did not replicate in direction or magnitude. Variation across observations is attributed to the seeds that produced them, not subjects, because each observation is one seed. This distinction matters for generalization claims; we evaluated on people who appear nowhere in training, supporting a claim about unseen subjects, but not evidence about a different corpus, sensor, task or deployment distribution. The finding stands against specific cited papers by extending the null regularization results observed for input projections ^[2] to attention query projections. It agrees with work questioning the validity of Transformer-based solutions for time series analysis tasks where simpler models suffice ^[4]. Future work would settle the question by testing deeper architectures where layer hierarchy is more critical, or by varying the consistency coefficient. It would also test whether consistency helps in low-data regimes where overfitting is more severe. The inconclusive outcome owes the reader every one of those interpretations and is never rewritten as a positive finding. The results clarify that for standard 2-block transformers on human activity recognition, cross-layer query consistency is not a sufficient inductive bias for subject-disjoint generalization.

This study has several limitations. First, we evaluated on a single dataset (UCI HAR) with short sequence lengths. Results may differ on longer time series where deep hierarchical processing is more beneficial. Second, the architecture was limited to 2 blocks; cross-layer consistency might matter more in deeper networks where divergence between layers is greater. Third, we tested only query projection consistency; key, value, or output projections might behave differently. Fourth, the regularization coefficient was fixed at 0.01; a hyperparameter search might find a regime where consistency helps. Finally, while we used subject-disjoint splits, we did not test cross-domain generalization (e.g., different sensor types). These limitations constrain the generalizability of the null result to other time-series benchmarks. Declared scope. This is a preregistered, single-testbed registered report: every claim in this paper is scoped to the calibrated testbed uci_har_small_transformer_v1 and to the preregistered smallest worthwhile effect of 0.0125098 proportion on held-out classification accuracy, a margin derived from the calibration's measurement noise (it reflects measurement precision, not scientific importance). Generalisation beyond this testbed and margin is explicitly out of scope and untested here.

This study tested whether cross-layer query consistency loss improves subject-disjoint generalization in time-series transformers. The results were inconclusive: the proposed condition did not exceed the preregistered worthwhile margin in either the primary or replication cohorts. Variation across observations is attributed to the seeds that produced them. For someone choosing between these methods, the evidence suggests standard training is sufficient for 2-block transformers on UCI HAR, as consistency loss adds complexity without accuracy gains. Future work should test deeper architectures and varying regularization strengths to settle whether layer-specific adaptation is essential or if consistency benefits other regimes.