{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DRLGyro\n",
    "\n",
    "DRLGyro is the project folder for my semester project *Deep Reinforcement Learning to Control a Gyroscope*\n",
    "conducted during the 2020 spring semester at EPFL. \n",
    "\n",
    "For more information, please contact me at: <lecauchoismatthieu@gmail.com>\n",
    "\n",
    "## Installation\n",
    "\n",
    "The project highly depends on two frameworks, namely OpenAI's *Gym* and *Spinning Up* frameworks, which can be installed as such:\n",
    "\n",
    "- For Gym:\n",
    "\n",
    "```pip install gym ```\n",
    "\n",
    "- For Spinning Up:\n",
    "\n",
    "```git clone https://github.com/openai/spinningup.git \n",
    "cd spinningup \n",
    "pip install -e . ```\n",
    "\n",
    "## Files\n",
    "\n",
    "The project folder consists of two main directories:\n",
    "```bash\n",
    "├── code 'All code files and model files of the project'\n",
    "└── report 'Report and pictures of the report'\n",
    "```\n",
    "\n",
    "The *code* directory contains the utilities, experiments and results of the project. Not all directories and sub-directories are shown here, such as the many model folders that were generated during the experimentation process, but the most relevant ones are. To understand what the many model experimentation folders correspond to, please refer to the notebooks where the experiments are performed. \n",
    "```bash\n",
    "├── custom_functions 'Custom python library implementing useful functions for the project'\n",
    "├── environment\n",
    "│   ├── Gyroscope 'Environment folder ready for import as a library'\n",
    "│   │   └── gym_GyroscopeEnv\n",
    "│   │       └── envs\n",
    "│   │           ├── gyroscopediscontinuous_env.py 'Environment with discontinuity'\n",
    "│   │           ├── gyroscope_env.py 'Simple environment'\n",
    "│   │           ├── gyroscopeintegral_env.py 'Environment with additionam integral term'\n",
    "│   │           └── gyroscoperobust_env.py 'Robust environment'\n",
    "│   └── gyroscope_environment_testing.ipynb 'Simulation code (python class) transferred to the Gym API'\n",
    "├── simulation 'Initial simulation experiments'\n",
    "│   ├── AgramFrame.png\n",
    "│   └── gyroscope_simulation.ipynb 'Simulation coding and experiments'\n",
    "├── training_spinuplib 'Experiments using the spinning up library'\n",
    "│   ├── activations_visualization.ipynb 'Visualization notebook to cluster activations using t-SNE'\n",
    "│   ├── baseline_norm_ddpg_td3_sac.ipynb 'Testing notebook between DRL algorithms for norm. reward'\n",
    "│   ├── baseline_quadreward_ddpg_td3_sac.ipynb 'Testing notebook between DRL algorithms for quad. reward'\n",
    "│   ├── discontinuity_ddpg.ipynb 'Testing notebook for discontinuous environment'\n",
    "│   ├── envvariants_quadreward.ipynb 'Testing notebook with additional integral term'\n",
    "│   ├── paramsearch_iter0_norm_td3.ipynb 'Testing notebook for hyperparamater search iteration 0'\n",
    "│   ├── paramsearch_iter1_norm_td3.ipynb 'Testing notebook for hyperparamater search iteration 1'\n",
    "│   ├── paramsearch_iter2_norm_td3.ipynb 'Testing notebook for hyperparamater search iteration 2 and 3'\n",
    "│   ├── reward_testing_ddpg.ipynb 'Testing notebook for reward function analysis'\n",
    "│   ├── robust_agent_testing.ipynb 'Testing notebook for the robust agent'\n",
    "│   ├── simple_agent_testing.ipynb 'Testing notebook for the simple agent'\n",
    "│   ├── td3_n_it2_an_var1_extralay_extralay_s0 'Model folder (load w/ pytorch) of the final simple agent'\n",
    "│   └── best_agent_robustified 'Model folder (load w/ pytorch) of the final robust agent'\n",
    "└── training_udalib 'Preliminary experiments using the udalib library before moving to spinning up'\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "drlgyroenv",
   "language": "python",
   "name": "drlgyroenv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}