The calculation of the elements of the jacobian matrix requires the values of the derivatives of the model function at the input data points. TCAD model functions are generally smooth but their analytical derivatives are unavailable. Numerical differentiations techniques are used to approximate the derivative using forward differences formulae:
or a central difference ones:
where 
is the unit vector in the 
direction and 
represents a small increment.
It was proven that the use of numerical approximation in the
Levenberg-Marquradt algorithm does not jeopardize its convergence
properties [16]. It is also known that numerical differentiation is an
unstable procedure prone to truncation and subtractive cancellation
errors [30]. Decreasing the step size () will reduce
the truncation error. Unfortunately a smaller step has the opposite effect
on the cancellation error. Selecting the optimal step size for a
certain problem is computationally expensive [30] and the
benefits achieved are not justifiable as the effect of small errors in the
values of the elements of the Jacobian matrix is minor. For this reason,
the sizing of the finite difference step
is not attempted and a constant increment size is used in evaluating the
gradient. Furthermore, to allow for the flexibility to adjust to the specific
problem characteristics, the individual size of each parameter step can be
adjusted by the user.
Calculation of the numerical approximation of the Jacobian matrix requires
model evaluations for the forward difference case and 
for
central differences. To alleviate the computational burden, forward
differences are always used except when they become unreliable. This usually
happens near the solution when 
is small, or when the
difference in the values of the function is small relative to the step size.
In both cases, the central difference formulae is used instead. Finally,
a first-order Broyden update scheme is implemented and can also be used
to reduce the number of model function computations in the calculation of
the approximate Jacobian at the 
iteration as follows:
This scheme requires only 
functional evaluation and can be used up to
times in sequence after each evaluation of the Jacobian matrix by forward
or central differences. A note of caution is due here: For highly nonlinear
or poorly conditioned models, the errors encountered in using the Broyden
update scheme could cause larger parameter deviation which could result in
the trapping of the solution at a local minimum.