Second derivatives of mathematical models for real-world phenomena are fundamental ingredients of a wide range of numerical simulation methods including parameter sensitivity analysis, uncertainty quantification, nonlinear optimization and model calibration. The evaluation of such Hessians often dominates the overall computational effort. Various combinatorial optimization problems can be formulated based on the highly desirable exploitation of the associativity of the chain rule of differential calculus. The fundamental Hessian Accumulation problem aiming to minimize the number of floating-point operations required for the computation of a Hessian turns out to be NP-complete. The restriction to suitable subspaces of the exponential search space proposed in this paper ensures computational tractability while yielding improvements by factors of ten and higher over standard approaches based on second-order tangent and adjoint algorithmic differentiation. Motivated by second-order parameter sensitivity analysis of surrogate numerical models obtained through training and pruning of deep neural networks this paper focusses on bracketing of dense Hessian chain products with the aim of minimizing the total number of floating-point operations to be performed. The results from a given dynamic programming algorithm for optimized bracketing of the underlying dense Jacobian chain product are used to reduce the computational cost of the corresponding Hessian. Minimal additional algorithmic effort is required.

中文翻译：

---

密集黑森式链括号的动态编程启发式

现实世界中数学模型的二阶导数是各种数值模拟方法的基本要素，包括参数敏感性分析，不确定性量化，非线性优化和模型校准。对这种黑森州人的评估通常支配着整个计算工作。可以基于对微积分的法则的关联性的高度期望的利用来制定各种组合优化问题。旨在最小化计算Hessian所需的浮点运算数量的基本Hessian累积问题证明是NP完全的。本文提出的对指数搜索空间的合适子空间的限制确保了计算的可处理性，同时在基于二阶切线和伴随算法微分的标准方法上，比标准方法提高了十倍或十倍以上。出于对通过深度神经网络的训练和修剪而获得的替代数值模型进行二阶参数敏感性分析的动机，本文将重点放在密集的Hessian链产品的括号中，目的是最大程度地减少要执行的浮点运算的总数。从给定的动态编程算法得到的结果，该算法用于对基础密集的Jacobian链乘积进行优化包围，以减少相应Hessian的计算成本。需要最少的额外算法工作量。