Minimising forecast error requires accurately specifying the initial state from which 
the forecast is made by optimally using available observing resources to obtain the 
most accurate possible analysis. The Kalman filter accomplishes this for linear sys-
tems and experience shows that the extended Kalman filter also performs well in 
non-linear systems. Unfortunately, the Kalman filter and the extended Kalman filter 
require computation of the time dependent error covariance matrix which presents 
a daunting computational burden. However, the dynamically relevant dimension of 
the forecast error system is generally far smaller than the full state dimension of the 
forecast model which suggests the use of reduced order error models to obtain near 
optimal state estimators. A method is described and illustrated for implementing a 
Kalman filter on a reduced order approximation of the forecast error system. This 
reduced order system is obtained by balanced truncation of the Hankel operator repre-
sentation of the full error system. As an example application a reduced order Kalman 
filter is constructed for a time-dependent quasi-geostrophic storm track model. The 
accuracy of the state identification by the reduced order Kalman filter is assessed 
and comparison made with the state estimate obtained by the full Kalman filter and 
with the estimate obtained using an approximation to 4D-Var. The accuracy assess-
ment is facilitated by formulating the state estimation methods as observer systems. 
A practical approximation to the reduced order Kalman filter that utilises 4D-Var 
algorithms is examined

Synoptic-scale eddy variance and fluxes of heat and momentum in midlatitude jets are sensitive to small changes in mean jet velocity, dissipation, and static stability. In this work the change in the jet producing the greatest increase in variance or flux is determined. Remarkably, a single jet structure change completely characterizes the sensitivity of a chosen quadratic statistical quantity to modification of the mean jet in the sense that an arbitrary change in the jet influences a chosen statistical quantity in proportion to the projection of the change on this single optimal structure. The method used extends previous work in which storm track statistics were obtained using a stochastic model of jet turbulence.

A generalized stability theory (GST) analysis of baroclinic shear flow is performed using primitive equations (PEs), and typical synoptic-scale midlatitudinal values of vertical shear and stratification. GST is a comprehensive linear stability theory that subsumes modal stability theory and extends it to account for nonmodal interactions such as may occur among the approximately geostrophically balanced modes and the nearly divergent gravity wave modes supported by the PE. Unbounded constant shear flow and the Eady model are taken as examples and energy is used as the reference norm. Comparison is made with results obtained using quasigeostrophic (QG) analysis. While the PE and the QG dynamics give similar results for timescales of a few days, the PE initial growth rate during the first few hours exceeds the QG growth at all wavelengths and can attain values as much as two orders of magnitude greater as the wavelength decreases. This PE growth is due both to the direct kinetic energy growth mechanism, which is filtered out by QG, and to the interaction between the QG modes and the gravity waves.

An important application of GST is to study shear turbulence using a method of analysis based on stochastically forcing the mean state. The PE response of the Eady model to spatially and temporally uncorrelated forcing reveals the rotational and the divergent modes support a comparable amount of variance. The observed spectrum and physical mechanisms influencing the spectrum are discussed.

Turbulence in fluids is commonly observed to coexist with relatively large spatial and temporal scale coherent jets. These jets may be steady, vacillate with a definite period, or be irregular. A comprehensive theory for this phenomenon is presented based on the mutual interaction between the coherent jet and the turbulent eddies. When a sufficient number of statistically independent realizations of the eddy field participate in organizing the jet a simplified asymptotic dynamics emerges with progression, as an order parameter such as the eddy forcing is increased, from a stable fixed point associated with a steady symmetric zonal jet through a pitchfork bifurcation to a stable asymmetric jet followed by a Hopf bifurcation to a stable limit cycle associated with a regularly vacillating jet and finally a transition to chaos. This underlying asymptotic dynamics emerges when a sufficient number of ensemble members is retained in the stochastic forcing of the jet but a qualitative different mean jet dynamics is found when a small number of ensemble members is retained as is appropriate for many physical systems. Example applications of this theory are presented including a model of midlatitude jet vacillation, emergence and maintenance of multiple jets in turbulent flow, a model of rapid reorganization of storm tracks as a threshold in radiative forcing is passed, and a model of the quasi-biennial oscillation. Because the statistically coupled wave-mean flow system discussed is generally globally stable this system also forms the basis for a comprehensive theory for equilibration of unstable jets in turbulent shear flow.

In studies of perturbation dynamics in physical systems, certain specification of the gov-
erning perturbation dynamical system is generally lacking, either because the perturba-
tion system is imperfectly known or because its specification is intrinsically uncertain, 
while a statistical characterization of the perturbation dynamical system is often avail-
able. In this report exact and asymptotically valid equations are derived for the ensemble 
mean and moment dynamics of uncertain systems. These results are used to extend the 
concept of optimal deterministic perturbation of certain systems to uncertain systems. 
Remarkably, the optimal perturbation problem has a simple solution: In uncertain sys-
tems there is a sure initial condition producing the greatest expected second moment 
perturbation growth.

Perturbation growth in uncertain systems associated with fluid flow is examined concentrating on deriving, solving, and interpreting equations governing the ensemble mean covariance. Covariance evolution equations are obtained for fluctuating operators and illustrative physical examples are solved. Stability boundaries are obtained constructively in terms of the amplitude and structure of operator fluctuation required for existence of bounded second-moment statistics in an uncertain system. The forced stable uncertain system is identified as a primary physical realization of second-moment dynamics by using an ergodic assumption to make the physical connection between ensemble statistics of stable stochastically excited systems and observations of time mean quantities. Optimal excitation analysis plays a central role in generalized stability theory and concepts of optimal deterministic and stochastic excitation of certain systems are extended in this work to uncertain systems. Re- markably, the optimal excitation problem has a simple solution in uncertain systems: there is a pure structure producing the greatest expected ensemble perturbation growth when this structure is used as an initial condition, and a pure structure that is most effective in exciting variance when this structure is used to stochastically force the system distributed in time.

Optimal excitation analysis leads to an interpretation of the EOF structure of the covariance both for the case of optimal initial excitation and for the optimal stochastic excitation distributed in time that maintains the statistically steady state. Concepts of pure and mixed states are introduced for interpreting covariances and these ideas are used to illustrate fundamental limitations on inverting covariances for structure in stochastic systems in the event that only the covariance is known.

Perturbation growth in uncertain systems is examined and related to previous work in which linear stability 
concepts were generalized from a perspective based on the nonnormality of the underlying linear operator. In 
this previous work the linear operator, subject to an initial perturbation or a stochastic forcing distributed in 
time, was either fixed or time varying, but in either case the operator was certain. However, in forecast and 
climate studies, complete knowledge of the dynamical system being perturbed is generally lacking; nevertheless, 
it is often the case that statistical properties characterizing the variability of the dynamical system are known. 
In the present work generalized stability theory is extended to such uncertain systems. The limits in which 
fluctuations about the mean of the operator are correlated over time intervals, short and long, compared to the 
timescale of the mean operator are examined and compared with the physically important transitional case of 
operator fluctuation on timescales comparable to the timescales of the mean operator. Exact and asymptotically 
valid equations for transient ensemble mean and moment growth in uncertain systems are derived and solved. 
In addition, exact and asymptotically valid equations for the ensemble mean response of a stable uncertain 
system to deterministic forcing are derived and solved. The ensemble mean response of the forced stable uncertain 
system obtained from this analysis is interpreted under the ergodic assumption as equal to the time mean of the 
state of the uncertain system as recorded by an averaging instrument. Optimal perturbations are obtained for 
the ensemble mean of an uncertain system in the case of harmonic forcing. Finally, it is shown that the remarkable 
systematic increase in asymptotic growth rate with moment in uncertain systems occurs only in the context of 
the ensemble.

Minimizing forecast error requires accurately specifying the initial state from which the forecast is made by 
optimally using available observing resources to obtain the most accurate possible analysis. The Kalman filter 
accomplishes this for a wide class of linear systems, and experience shows that the extended Kalman filter also 
performs well in nonlinear systems. Unfortunately, the Kalman filter and the extended Kalman filter require 
computation of the time-dependent error covariance matrix, which presents a daunting computational burden. 
However, the dynamically relevant dimension of the forecast error system is generally far smaller than the full 
state dimension of the forecast model, which suggests the use of reduced-order error models to obtain near-
optimal state estimators. A method is described and illustrated for implementing a Kalman filter on a reduced-
order approximation of the forecast error system. This reduced-order system is obtained by balanced truncation 
of the Hankel operator representation of the full error system and is used to construct a reduced-order Kalman 
filter for the purpose of state identification in a time-dependent quasigeostrophic storm track model. The accuracy 
of the state identification by the reduced-order Kalman filter is assessed by comparison to the true state, to the 
state estimate obtained by the full Kalman filter, and to the state estimate obtained by direct insertion.

Methods for approximating a stable linear autonomous dynamical system of lower order are examined. Reducing the order of a dynamical system is useful theoretically in identifying the irreducible dimension of the dynamics and in isolating the dominant spatial structures supporting the dynamics, and practically in providing tractable lower-dimension statistical models for climate studies and error covariance models for forecast analysis and initialization. Optimal solution of the model order reduction problem requires simultaneous representation of both the growing structures in the system and the structures into which these evolve. For autonomous operators associated with fluid flows a nearly optimal solution of the model order reduction problem with prescribed error bounds is obtained by truncating the dynamics in its Hankel operator representation. Simple model examples including a reduced-order model of Couette flow are used to illustrate the theory. Practical methods for obtaining approximations to the optimal order reduction problem based on finite-time singular vector analysis of the propagator are discussed and the accuracy of the resulting reduced models evaluated.

This note focuses on thermodynamic changes caused by Eastern Mediterranean (EM) subsidence anomalies. Subsidence anomalies are shown to modulate EM-wide stability with respect to moist ascent. Additionally, convective available potential energy (CAPE) generation rates, as well as mean CAPE, change coherently during extreme EM rainfall anomalies. It is suggested that the resulting modulation of convective rain generation is the process directly responsible for the observed rainfall anomalies.

This paper presents a simple theory for the association between observed eastern Mediterranean (EM) rainfall anomalies and North Atlantic (NA) climate variability. Large-scale NA atmospheric mass rearrangements, pri- marily a modulation of the Icelandic low and the subtropical high pressure systems, tend to extend beyond the NA. A particularly strong such teleconnection exists between the northern NA and southern Europe and the Mediterranean Basin. Pressure anomalies over Greenland-Iceland are thus associated with reversed-polarity anomalies centered over the northern Adriatic, affecting the entire Mediterranean Basin; elevated Greenland pressure is accompanied by an anomalous cyclone over the Mediterranean, and a Mediterranean high pressure system is present when pressure over Greenland is reduced. In the EM, these anomalies result in anomalous southerlies during Greenland highs, and northerlies during Greenland lows. Eastern Mediterranean southerlies warm the EM, while northerlies cool locally. Because heat advection by horizontal and vertical motions dominate the EM thermodynamic equation (or, put differently, because thickness advection dominates the omega equation), cooling by northerly winds results in enhanced subsidence. Conversely, warming by anomalous EM southerlies produces enhanced ascent. These vertical motion anomalies modify the stability of the mean column, resulting in the observed EM rainfall anomalies.