Neural Network Architecture Series

This series explores how neural network architecture evolved from residual connections to modern experiments with dense links, transformers, and automated design. Each post builds on the previous one while still standing on its own.

Series Progress

All Posts in This Series

Part 1: Artificial Neural Networks 17 Dec, 2021 Artificial neural networks (ANNs) are the cornerstone of Deep Learning algorithms. The name and the architecture are adopted from the human brain's neural network. ANNs are designed to simulate human reasoning based on how neurons communicate. ANNs contain a set of artificial neurons connected.
Part 3: Dense Connections and Information Flow: From DenseNet to Modern Variants Coming Soon Explore dense connectivity patterns, why they improve gradient flow, and when dense connections outperform residual links. This post is currently being written and will be published soon.
Part 6: Neural Architecture Search: Automating the Discovery of Better Connections Coming Soon How NAS explores connection patterns and what it reveals about architecture design. This post is currently being written and will be published soon.
Part 7: Bottlenecks, Inverted Residuals, and Mobile Architectures Coming Soon How mobile architectures adapt residual designs to fit edge devices. This post is currently being written and will be published soon.
Part 8: The Future of Connections: From Manifold Constraints to Dynamic Routing Coming Soon A survey of emerging connection patterns and where architecture research may head next. This post is currently being written and will be published soon.
Part 9: Transformers in Practice: From Architecture to a Chatbot Implementation Coming Soon A practical bridge from transformer architecture concepts to implementation: build and evaluate a chatbot workflow with TensorFlow and HuggingFace, with clear trade-offs and deployment guidance. This post is currently being written and will be published soon.

Getting Started

New to this series? Start with Part 1: Artificial Neural Networks.

Each post builds on the previous one, so reading them in order is recommended. Each post also works on its own if you need to jump to a specific topic.