01
Energy Landscape Flattening Techniques for Improved Neural Generalization
Karthikeyan Palanisamy, Suja M , Kanmani Natarajan · Pages 1–12
Deep learning has seen tremendous progress over recent years and has produced impressive results in areas like computer vision, natural language processing, and healthcare analytics. However, one of the key difficult problems with neural networks is a guarantees on seen generalization, where a model trained on some data does well even unseen data. The geometry of the loss surface, commonly called the energy landscape, is another important ingredient determining generalization. Inevitably, a growing body of work suggests that the flatness/sharpness of minima in the energy landscape correlates well against generalization ability: sharper minima (longer-trapped) optimise further and are even weaker on robustness than on overfitting. This has led to techniques that flatten the energy landscape with the hope of directing optimization algorithms to flatter and broader regions of the loss surface.Flattening of the energy landscape means change of training dynamics, objective function or optimization procedure to accelerate convergence towards flat minima.
Keywords: Energy Landscape, Generalization in Neural Networks, Optimization of Deep Learning, Flat Minima, Sharp Minimum—Regularization Methods, Sharpness-Aware Minimization (SAM), Loss Surface Analysis and Robustness Over fitting.
Article page pending
02
Geometric and Statistical Analysis of Neural Collapse in Deep Classifiers
Arik Elijah · Pages 1–12
Recently, Neural Collapse has been proposed as a 1st-order universal and interesting behavior occurring during the end phase of training deep neural networks in classification settings. Although the modern architectures that take deep learning to its high complexity and overparameterization are very heavily structured, empirical investigations show the trained models converge to very structured symmetries in feature space. In this paper, we provide a simple yet comprehensive geometric and statistical analysis of Neural Collapse in deep classifiers that aims to bridge the gap between empirical observations and theoretical understanding.
Keywords: Deep Learning, Representation Learning, Neural Collapse, Feature Geometry, Simplex ETF, Cross-Entropy Loss and Gradient Descent; Statistical Analysis of Generalization
Article page pending
03
Distribution ally Robust Optimization for Algorithmic Fairness in AI Systems
Anurak Doryu, Daiki Tae · Pages 1–12
Massive deployment of Artificial Intelligence (AI) systems in various vital fields like healthcare, finance, education and hiring has raised pressing issues about fairness, bias and ethical decision-making. Since machine learning models always learn from historical data, they unintentionally reflect our societal biases to the extent they are present in the collected datasets; thus leading to unfair bias against underrepresented or disadvantaged classes. By the same token, solving these challenges entails not solely enhancing models for accuracy but making them robust and fair under uncertain and changing data distributions.
Keywords: Distribution Ally Robust Optimization, Algorithmic Fairness, Bias Mitigation In Learning Models, Ethical AI Principles, Machine Learning Algorithms, Fairness Metrics For Machine Learning Methods, Robust Learning Under Distribution Shift Types Of Data Distortion; AI Ethics Techniques And Approaches Keywords
Article page pending
04
Graph Signal Processing for Learning on Non-Euclidean Data Structures
KM Syed Arif, Shuba Shree, Dr. Shva Shankari · Pages 1–12
Graph-structured data has become increasingly prevalent across a wide range of modern applications, including social networks, biological systems, transportation networks, and recommendation systems. Unlike traditional Euclidean data, which resides on regular grids such as images or time series, graph data is inherently irregular and complex, characterized by non-Euclidean relationships among entities. This irregularity poses significant challenges for conventional machine learning and signal processing techniques, which rely heavily on structured domains.
Keywords: Graph Signal Processing, Non-Euclidean Data, Graph Fourier Transform, Spectral Graph Theory, Graph Neural Networks, Graph Filtering, Deep Learning On Graphs, Network Data Analysis, Signal Representation, Graph-Based Learning
Article page pending
05
Physics-Informed Neural Networks for Solving Nonlinear Partial Differential Equations
Sukanya Shree, Sathya Gurunathan, Suguna Kumari · Pages 1–12
In the field of scientific computing, recent work has studied Physics Informed Neural Networks (PINNs) as a novel computational paradigm for weakly supervised learning-based methods to solve nonlinear partial differential equations (PDEs), bridging progress in classical numerical analysis with recent advances in general machine learning. Traditional numerical methods such as the Finite Difference Method (FDM), the Finite Element Method (FEM) and spectral approaches, which have been around for decades to numerically solve partial differential equations modelling a physical system. However, their performance suffers from limitations such as mesh dependency and high computational cost in higher dimensional domains, and they struggle to represent complicated geometries or sparse data scenarios.
Keywords: Physics-Informed Neural Networks, Nonlinear Partial Differential Equations, Deep Learning, Scientific Machine Learning, Automatic Differentiation, Variation Methods, Neural Operators, Computational Physics
Article page pending
