Mingxuan Li

Mingxuan Li

MLE | AI4Science | Drug Discovery

Literature Notes - Physics-informed machine learning

1 minute read

Published:

The text discusses the advancements in Physics-Informed Neural Networks (PINNs) facilitated by their integration with modern machine learning libraries such as TensorFlow, PyTorch, and specialized frameworks like DeepXDE and SimNet. PINNs excel in incorporating physical laws into machine learning models, making them particularly adept at handling complex, multidimensional scientific problems.

PINNs Framework

Foundations and Inspirations of Physics-Informed ML

Physics-Informed Machine Learning Methods

Applications of Physics-Informed ML

Thoughts on PINNs and ML Frameworks

  1. Integration with Classical Methods:
    • PINNs effectively integrate with classical numerical methods like finite element methods (FEM), boundary element methods (BEM), and finite volume methods (FVM), enhancing their applicability in scientific computing.
  2. Framework Utilization:
    • Libraries such as DeepXDE and SimNet simplify the application of PINNs by handling complex differential equations and integrating soft penalty constraints that embed physical laws into the training process.
  3. Scalability and Performance:
    • Hybrid models that combine PINNs with domain decomposition techniques enable scalability and efficient computation across large-scale simulations, supported by parallel computing frameworks.
  4. Advanced Architectures:
    • The use of meta-learning techniques to automate the configuration of PINNs models presents a forward-looking approach that potentially lowers the barrier to entry for complex model setups.
  5. Future Directions:
    • Enhanced Computational Methods: Further integration with cutting-edge computational techniques such as machine learning-optimized tensor operations could enhance the efficiency of PINNs.
    • Educational Applications: Given their structured and intuitive setup, PINNs frameworks are well-suited for educational purposes in computational science and engineering fields.

You May Also Enjoy

Literature Notes - Geneformer

3 minute read

Published:

  • Traditional methods for analyzing single-cell transcriptomics rely on cell-type-specific annotations or perturbation-based studies, which can be resource-intensive and dataset-specific.
  • Geneformer, a transformer-based foundation model, learns transcriptional dynamics directly from large-scale unlabelled single-cell transcriptomes, enabling context-aware gene mapping across diverse cellular states and conditions.
  • Pretrained on Genecorpus-30M (29.9 million single-cell transcriptomes), Geneformer provides a robust framework for a wide range of downstream applications, from dosage sensitivity prediction to in silico gene perturbation analysis.

Literature Notes - ESM3

1 minute read

Published:

  • ESM3 pushes the boundaries of protein design by integrating sequence, structure, and function into a single unified framework.
  • It scales up to 98 billion parameters and leverages multimodal inputs, enabling generative design and evolutionary insights.
  • Simulates evolutionary processes, producing proteins equivalent to 500 million years of natural evolution within days.

Literature Notes - ESM2

1 minute read

Published:

  • Traditional protein prediction models rely heavily on evolutionary information, often requiring computationally expensive multiple sequence alignment (MSA) inputs.
  • ESM2, powered by large language models (LLMs), learns evolutionary patterns directly from raw protein sequences, eliminating MSA requirements and simplifying the computational pipeline.
  • Achieves a ~60x faster inference compared to prior state-of-the-art methods, facilitating studies on vast metagenomic datasets (e.g., MGnify90).

Literature Notes - MolFormer

1 minute read

Published:

The study highlights the capabilities of the Molecule Transformer (M-Transformer), an unsupervised, pretrained molecular language model that excels in predicting molecular properties from SMILES sequences. This model surpasses traditional graph-based models in various benchmarks, efficiently utilizes computational resources by reducing GPU usage by a factor of 60, and accurately captures interatomic relationships. Further exploration into expanding its applicability beyond small organic molecules is recommended.