LSTM error with number of X and Y observations

The MATLAB LSTM error stating that the number of observations in X and Y must match occurs when the outer dimensions of your training cell arrays differ (meaning numel(X) ~= numel(Y)) or when time-series shifting introduced an off-by-one index mismatch during lag creation.

Primary Root Causes and How to Fix Them

1. Off-By-One Lagging Mismatch: When preparing time-series forecasting data, developers often lag the target vector using X = data(1:end-1) and Y = data(2:end). If done across multiple sequence files or batch matrices without synchronizing row counts, numel(X) will not match numel(Y).
2. Cell Array vs. Matrix Format: For variable-length sequences or multi-channel time series, MATLAB trainNetwork() expects X to be an N-by-1 cell array, where each cell contains a numeric matrix of size [numFeatures, numTimeSteps]. Passing a 2D or 3D numeric array can cause MATLAB to misinterpret the observation dimension.
3. Sequence-to-Sequence vs. Sequence-to-One Conflict:
   • For Sequence-to-One: Y must be a categorical array (classification) or numeric matrix (regression) with exactly N rows.
   • For Sequence-to-Sequence: Y must be an N-by-1 cell array where each cell Y{i} has the exact same number of time steps as X{i} (meaning size(X{i}, 2) == size(Y{i}, 2)).

Executable MATLAB Code: Correct Data Formatting for LSTM

% =========================================================================
% Resolving Observation Dimension Mismatch in MATLAB LSTM Networks
% =========================================================================

% 1. Create synthetic multi-step time series data
totalPoints = 1000;
timeData = sin(linspace(0, 50, totalPoints));

% 2. Partition into sequences using a sliding window
sequenceLength = 50;
stepAhead = 1;

numSequences = totalPoints - sequenceLength - stepAhead + 1;

% Pre-allocate cell arrays for X and Y
XTrain = cell(numSequences, 1);
YTrain = cell(numSequences, 1); % Sequence-to-sequence target

for i = 1:numSequences
    % Features along rows (1 feature), time steps along columns (sequenceLength)
    seqX = timeData(i : i + sequenceLength - 1);
    seqY = timeData(i + stepAhead : i + sequenceLength + stepAhead - 1);
    
    XTrain{i} = seqX; % Size: [1, 50]
    YTrain{i} = seqY; % Size: [1, 50]
end

% 3. Verification assertion: prevent the error before training
assert(numel(XTrain) == numel(YTrain), ...
    'Dimension Error: Outer observation count of X and Y must be identical.');
assert(size(XTrain{1}, 2) == size(YTrain{1}, 2), ...
    'Sequence Error: Number of time steps in X and Y must match for sequence-to-sequence.');

fprintf('Data validated successfully. Total observations: %d\n', numel(XTrain));

% 4. Build a standard LSTM architecture
numFeatures = 1;
numHiddenUnits = 64;
numResponses = 1;

layers = [
    sequenceInputLayer(numFeatures, 'Name', 'input')
    lstmLayer(numHiddenUnits, 'OutputMode', 'sequence', 'Name', 'lstm')
    fullyConnectedLayer(numResponses, 'Name', 'fc')
    regressionLayer('Name', 'output')
];

options = trainingOptions('adam', ...
    'MaxEpochs', 15, ...
    'MiniBatchSize', 32, ...
    'InitialLearnRate', 0.005, ...
    'Shuffle', 'every-epoch', ...
    'Plots', 'training-progress', ...
    'Verbose', false);

% 5. Execute training without dimension errors
net = trainNetwork(XTrain, YTrain, layers, options);

Pre-Training Diagnostic Checklist

• Verify Dimensions: Run size(X) and size(Y) in your Command Window. If X is a cell array of 100 elements, numel(Y) must equal exactly 100.
• Check Feature Orientation: Inside each cell, rows must represent features and columns must represent time steps. If your data is [timeSteps, features], apply transpose X{i} = X{i}'.
• Sequence-to-One Conversion: If you only want to predict the single next value after each sequence, change the layer parameter to lstmLayer(numHiddenUnits, 'OutputMode', 'last') and convert YTrain from a cell array into a numeric column vector [numObservations, 1].