I have an equation given by:

$$ \frac{\mathrm{d} s}{\mathrm{d} t}=4a−2s+\lambda(s) $$

where, $a$ is an input constant and $\lambda$ is a non-linear term that depends on $s$.

I know that the true solution for $\lambda\left(s\right)$ is $\sin \left( s\right) \cos\left(s\right)$

I have generated data from [0, 5]s with the true solution and with the equation excluding the non-linear term for a range of initial conditions and a range of $a$'s.

import numpy as np
import pandas as pd
from scipy.integrate import solve_ivp

def foo_true(t, s, a):
    ds_dt = 4*a -2*s + np.sin(s)*np.cos(s)
    return ds_dt

def foo(t, s, a):
    ds_dt = 4*a -2*s
    return ds_dt

# Settings:
t = np.linspace(0, 5, 1000) 

a_s = np.arange(1, 10)
s0_s = np.arange(1, 10)

# Store the data
df_true = pd.DataFrame({'time': t})
df = pd.DataFrame({'time': t})

# Generate the data
for a in a_s:
    for s0 in s0_s:
        sol_true = solve_ivp(foo_true, (t[0], t[-1]), (s0,), t_eval=t, args=(a,))
        df_true[f'{a}_{s0}'] = sol_true.y.T
for a in a_s:
    for s0 in s0_s:
        sol = solve_ivp(foo, (t[0], t[-1]), (s0,), t_eval=t, args=(a,))    
        df[f'{a}_{s0}'] = sol.y.T

In the above code, df_true is a data frame containing the actual dynamics of the system and df is the dynamics with a discrepancy.

The figure below shows an example of the discrepancy in the data:

enter image description here

Given that I know part of the physics, how can I model $\lambda$ with pyTorch?

Can I look into a paper/repo with a conceptually similar example?


2 Answers 2


Yes. I was searching for the same thing a while back and I came across the concept of PINNs.

Physics-informed neural networks (PINNs) are neural networks that encode model equations, such as partial differential equations (PDEs), as a component of the neural network itself.

enter image description here

This article describes PINNs for a harmonic oscillator! Code for the harmonic oscillator problem.

This link describes the concept in detail with the help of a simple projectile motion example.


  • $\begingroup$ Hi, thanks for the input. This problem required to able to take in multiple solutions of the ODE and model the $\lambda$ term. Can PINNs take in multiple initial conditions? $\endgroup$
    – user149991
    Commented May 25, 2023 at 11:34

Modeling the non-linear term $λ(s)$ in your equation using PyTorch can be approached as a parameter estimation or function approximation problem. Since you know the true solution for $λ(s)$, you can use the generated data to train a neural network to approximate the non-linear term based on the given input and output values.

To model $λ(s)$ using PyTorch, you can treat it as a learnable parameter in a neural network and train the network to approximate the true $λ(s) = sin(s) * cos(s)$ based on the given data. Instead of using a single input $s$, you can use a concatenated input of $s$ and the corresponding solution values. This way, the model can learn the relationship between $λ(s)$ and the given solutions.

Here's an example of how you can implement this using PyTorch:

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, Dataset

# Define a custom dataset for the data
class CustomDataset(Dataset):
    def __init__(self, data):
        self.data = data

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        return self.data[idx]

# Define a neural network to approximate λ(s)
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.linear = nn.Linear(2, 1)  # Input dimension is 2 (s and solution)

    def forward(self, x):
        return torch.sin(x[:, 0]) * torch.cos(x[:, 0]) + self.linear(x)

# Prepare the data
data = df.values  # Include all columns
dataset = CustomDataset(data)
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)

# Create the model and optimizer
model = Model()
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.MSELoss()

# Training loop
num_epochs = 100
for epoch in range(num_epochs):
    for batch in dataloader:
        inputs = batch[:, :-1]  # Input data (exclude the last column)
        targets = batch[:, -1]  # True λ(s) values (last column)
        outputs = model(inputs)
        loss = criterion(outputs.squeeze(), targets)
    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {loss.item()}")

# Test the model
test_input = torch.tensor([[1.0, 0.0]])  # An example input (s=1.0, solution=0.0)
predicted_output = model(test_input)
print(f"Predicted λ(s) for input s=1.0: {predicted_output.item()}")

# Continue with your desired analysis or visualization using the predicted lambda values

This code defines a custom dataset class, a neural network model with a linear layer, sets up the optimizer and loss function, and trains the model using the provided data. The model approximates the true $λ(s)$ by combining the learned linear term with the sin(s)cos(s) term. The input dimension of the model is now 2, consisting of s and the corresponding solution value. The model's linear layer is adjusted accordingly to accommodate the concatenated input.

Regarding papers or repositories with similar examples, here are a few resources you can explore for inspiration:

  1. Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations by Raissi et al

    • This paper presents a physics-informed neural network approach for solving differential equations. It combines physical laws with data to train neural networks to approximate the unknown terms in the equations.
  2. Neural Ordinary Differential Equations by Chen et al.

    • This paper introduces Neural Ordinary Differential Equations (NODEs), which use continuous-depth models to approximate the solution of differential equations. It provides insights into how neural networks can be used to represent differential equations.
  3. Physics-Informed Neural Networks (PINNs): An Introduction and Recent Advances by Zhang et al.

    • This review paper introduces Physics-Informed Neural Networks (PINNs) and their applications in solving various physical problems. It discusses the integration of physics-based knowledge into neural networks.
    • You can explore the code and methodologies presented in these papers to gain a deeper understanding of modeling non-linear terms in physics using deep learning approaches.
  4. DeepXDE: A Deep Learning Library for Solving Differential Equations by Lu, Weinan, and Zhongqiang Zhang.

These references provide examples and implementations of neural network-based approaches for solving differential equations and may serve as a starting point for your work on modelling $λ(s)$ with PyTorch.

Remember that modelling $λ(s)$ using PyTorch is an iterative process that may require fine-tuning the network architecture, hyperparameters, and training procedure to achieve the best results for your specific problem.

  • $\begingroup$ Thanks for the reply. The code produces an error. The shape of inputs is $[32, 81$ and the linear layers is expecting 2 entries $\endgroup$
    – user149991
    Commented May 26, 2023 at 9:08
  • $\begingroup$ could you help me with the shape problem? $\endgroup$
    – user149991
    Commented May 27, 2023 at 8:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.