• Login
    View Item 
    •   UMB Digital Archive
    • UMB Open Access Articles
    • UMB Open Access Articles
    • View Item
    •   UMB Digital Archive
    • UMB Open Access Articles
    • UMB Open Access Articles
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Browse

    All of UMB Digital ArchiveCommunitiesPublication DateAuthorsTitlesSubjectsThis CollectionPublication DateAuthorsTitlesSubjects

    My Account

    LoginRegister

    Statistics

    Display statistics

    Stiff neural ordinary differential equations

    • CSV
    • RefMan
    • EndNote
    • BibTex
    • RefWorks
    Author
    Kim, Suyong
    Ji, Weiqi
    Deng, Sili
    Ma, Yingbo
    Rackauckas, Christopher
    Date
    2021-09-20
    Journal
    Chaos
    Publisher
    American Institute of Physics Inc.
    Type
    Article
    
    Metadata
    Show full item record
    See at
    https://doi.org/10.1063/5.0060697
    Abstract
    Neural Ordinary Differential Equations (ODEs) are a promising approach to learn dynamical models from time-series data in science and engineering applications. This work aims at learning neural ODEs for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODEs in the classical stiff ODE systems of Robertson’s problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson’s problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODEs. The success of learning stiff neural ODEs opens up possibilities of using neural ODEs in applications with widely varying time-scales, such as chemical dynamics in energy conversion, environmental engineering, and life sciences. Neural Ordinary Differential Equations (ODEs) have been emerged as a powerful tool to describe a dynamical system using an artificial neural network. Despite its many advantages, there are many scientific and engineering examples where neural ODEs may fail due to stiffness. This study demonstrates how instability arises in neural ODEs during the training with the benchmark stiff system. Furthermore, we apply scaling to differential equations and loss functions to mitigate stiffness. The proposed technique in adjoint handling and equation scaling for stiff neural ODEs can be used in many biological, chemical, and environmental systems subject to stiffness. © 2021 Author(s).
    Keyword
    Neural Ordinary Differential Equations (ODEs)
    Stiff computation (Differential equations)
    Identifier to cite or link to this item
    http://hdl.handle.net/10713/16722
    ae974a485f413a2113503eed53cd6c53
    10.1063/5.0060697
    Scopus Count
    Collections
    UMB Open Access Articles

    entitlement

     
    DSpace software (copyright © 2002 - 2022)  DuraSpace
    Quick Guide | Policies | Contact Us | UMB Health Sciences & Human Services Library
    Open Repository is a service operated by 
    Atmire NV
     

    Export search results

    The export option will allow you to export the current search results of the entered query to a file. Different formats are available for download. To export the items, click on the button corresponding with the preferred download format.

    By default, clicking on the export buttons will result in a download of the allowed maximum amount of items.

    To select a subset of the search results, click "Selective Export" button and make a selection of the items you want to export. The amount of items that can be exported at once is similarly restricted as the full export.

    After making a selection, click one of the export format buttons. The amount of items that will be exported is indicated in the bubble next to export format.