Autonomous Vehicle Simulation

Self-driving car simulator using Unity and deep reinforcement learning with custom physics, traffic systems, and real-time computer vision integration.

This project implements a comprehensive self-driving car simulator using Unity ML-Agents with custom deep reinforcement learning algorithms. The system features realistic vehicle physics, dynamic traffic simulation, and integrates state-of-the-art computer vision models for lane and object detection in real-time.

Custom city environment with realistic vehicle physics simulating six distinct forces and dynamic traffic system.

Simulation Architecture

The simulator creates two training environments with custom car physics based on six simulated forces:

  • Suspension, Acceleration, Steering - Core vehicle dynamics
  • Brakes, Slipping, Friction - Realistic movement constraints

Animation curves were implemented for hyperrealistic vehicle feel and precise control. The traffic system utilizes a custom waypoint tool for flexible route planning with branching at intersections to increase environmental randomness.

Technical Implementation

Machine Learning Pipeline

  • Unity ML-Agents Release 20 with Python Low Level API
  • Custom DQN Algorithm for vehicle control (throttle/steering)
  • Sensor Integration: LIDAR and Camera data fusion
  • Computer Vision: CLRNet for lane detection and YOLOv8 for object detection
  • Learning from Demonstrations through experience replay memory

System Features

  • Dynamic traffic spawning using object pooling and navigation mesh
  • Custom side channels for real-time UI visualization
  • Checkpoint system for training progression
  • Multi-platform standalone builds
  • Real-time performance monitoring in TensorBoard
Neural network architecture for DQN implementation (left) and real-time visualization interface showing vehicle controls and sensor data (right).

Project Outcomes

  • Achieved 94% evaluation score and ranked as top graphics project for bachelor’s thesis
  • Successfully coordinated 6 researchers across computer vision, graphics, AI, and systems engineering
  • Implemented comparative analysis between LIDAR-only and LIDAR+CLRNet enhanced models
  • Developed custom reward function that significantly improved training efficiency
  • Average vehicle speed of 30 km/h using DQN control methods

Project Documentation: View Full Project PDF

Source Code: Available upon request at aveen2000hussein@gmail.com

Technologies Used

  • Unity 3D with ML-Agents
  • Python with PyTorch
  • C# for Unity scripting
  • YOLOv8 and CLRNet for computer vision
  • Deep Q-Network (DQN) reinforcement learning
  • Custom Physics Engine with six-force simulation