{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "EE-311\n", "======\n", "\n", "Lab 5: Dimensionality Reduction\n", "----------------------------------------\n", "\n", "created by Zahra Farsijani and François Marelli on 25.03.2020\n", "\n", "# Homework\n", "\n", "The file `homework.py` contains the homework of the week. It contains empty functions that must be completed according to the instructions.\n", "\n", "When the homework is completed, it must be submitted on Moodle for grading.\n", "\n", "**Do not change the function definitions in the file!**\n", "\n", "## Data generation\n", "\n", "PCA does not take the labels into account when computing the principal components of a dataset. This can lead to a dataset of reduced dimensionality with poor classification performance in some cases (fortunately, this is not frequent).\n", "\n", "In order to illustrate this effect, you are given a 2D dataset that is exactly linearly separable, but that becomes non separable when reduced to 1D using PCA. In that situation, using PCA actually decreases the performance of our model! Have a look at the data in the next cell, and how the distribution of classes can fool the PCA. Where is the axis of maximum variance? And which axis should we use for classification?" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGbCAYAAAAY8u5bAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAmkklEQVR4nO3df4wc5Z3n8c/X4zEZ48RjYJmAx4sJOCyGeHEyC7lFnGbiJJCNF2OUdRxdIuswQtkjgbAJCw7Cl0MXBY4THIjkJCtwIIF3YlhiSLIcAXtnT7eSw+IYTIB4wXABD2AgwV4Mk/jHfO+P7rF7pqq6u7qf7qrufr/+sae6purpb1dXffp5nuoxdxcAAADCmZZ1AwAAANoNAQsAACAwAhYAAEBgBCwAAIDACFgAAACBTc+6AaWOO+44nz9/ftbNaLj33ntPRx99dNbNyBVqEkVNoqhJFDWJoiZR1CQqRE22bt36trv/UdxjuQpY8+fP15NPPpl1MxpuZGREg4ODWTcjV6hJFDWJoiZR1CSKmkRRk6gQNTGz3yQ9xhAhAABAYAQsAACAwAhYAAAAgeVqDlac/fv3a+fOnXr//fezbkowH/zgB7V161ZJ0syZM3XKKadoxowZGbcKAACEkvuAtXPnTvX29uq0007TtGnt1eE2Pj6u3bt3a+fOnTr99NOzbg4AAAgk94nl/fffV19fX9uFK0maNm2a+vr62qp3DgAAtEDAktSW4WpCOz83AAA6FVd3AACAwAhYDfTyyy/rnHPO0YIFC/TFL35R+/fvz7pJAACgCQhYDXTNNdfoqquu0gsvvKA5c+bozjvvzLpJAACgCdouYG3cNqpzb9ysk6/9mc69cbM2bhute5vXX3+9brvttsM/X3fddbr99tvL/o67a/PmzfrCF74gSVq1apU2btxYd1sAAED+5f5rGtLYuG1Uax58RmMHDkmSRveMac2Dz0iSLlo8t+btrl69WhdffLGuvPJKjY+Pa3h4WJs3b9ZZZ50Vu/769et1/PHHq7e3V9OnF0rc39+v0dH6wx4AAMi/tgpYNz+643C4mjB24JBufnRHXQFr/vz5OvbYY7Vt2zbt3r1bixcv1kknnaSnnnoq8XfeeuutyDIzq7kNAACgdbRVwHptz1iq5Wlceumluvvuu/XGG2/okksu0bvvvqvzzjsvdt3169fr9NNP1549e3Tw4EFNnz5du3bt0oknnlh3OwAAQP61VcA6sbdHozFh6sTenrq3vXz5cq1du1YHDhzQ+vXr1dXVVbYHS5KGhob0wAMPaOXKlbrnnnu0bNmyutsBAADyr60muV99/mnq6e6atKynu0tXn39a3dueMWOGhoaGtGLFCnV1dVX+BUk33XSTbrnlFp166qn67W9/q9WrV9fdDgAAkH9t1YM1Mc/q5kd36LU9Yzqxt0dXn39aXfOvJoyPj2vLli26//77q/6dj3zkI3riiSfq3jcAAGgtbRWwpELIChGoSj333HNaunSpli9frgULFgTdNgAAaD9tF7AaYeHChXrppZeybgYAAGgRbTUHCwAAIA8IWAAAAIERsAAAAAIjYAEAAARGwGqgO+64Q6eeeqrMTG+//XbWzQEAAE1CwGqgc889V48//rhOOumkrJsCAACaqP0C1vYN0q1nSt/pLfy7fUPdm7z++ut12223Hf75uuuu0+23317x9xYvXqz58+fXvX8AANBa2ut7sLZvkH5yhXSg+PcI975a+FmSFq2oebOrV6/WxRdfrCuvvFLj4+MaHh7W5s2bddZZZ8Wuv379ei1cuLDm/QEAgNbWXgFr0w1HwtWEA2OF5XUErPnz5+vYY4/Vtm3btHv3bi1evFgnnXRSxT/2DAAAOlN7Bay9u9ItT+HSSy/V3XffrTfeeEOXXHKJ3n33XZ133nmx69KDBQBAZ2uvgDW7vzAsGLe8TsuXL9fatWt14MABrV+/Xl1dXfRgAQCAWO01yX3JWqm7Z/Ky7p7C8jrNmDFDQ0NDWrFihbq6uqr6ndtvv139/f3atWuXFi1apEsvvbTudgAAgPxrrx6siXlWm24oDAvO7i+EqzrmX00YHx/Xli1bdP/991f9O1dccYWuuOKKuvcNAABaS3sFLKkQpgIEqlLPPfecli5dquXLl2vBggVBtw0AANpP+wWsBli4cKFeeumlrJsBAABaREvMwRofH8+6CQ3Tzs8NAIBOlfuANXPmTO3evbstg8j4+Lh2796tmTNnZt0UAAAQUO6HCE855RTt3LlTr732WtZNaYiZM2fqlFNOyboZAAAgoNwHrBkzZuj000/PuhlBjYyMaHBwMOtmAACABsn9ECEAAECrIWABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAgsWsMysy8y2mdlPiz8fY2aPmdkLxX/nhNoXAABAnoXswbpS0vMlP18raZO7L5C0qfgzAABA2wsSsMysX9LnJf2wZPEySfcU/3+PpItC7AsAACDvzN3r34jZA5K+J+mDkr7l7kvNbI+795as8467R4YJzewySZdJUl9f3yeGh4frbk/e7du3T7Nmzcq6GblCTaKoSRQ1iaImUdQkippEhajJ0NDQVncfiHtsel1blmRmSyW96e5bzWww7e+7+zpJ6yRpYGDABwdTb6LljIyMqBOeZxrUJIqaRFGTKGoSRU2iqElUo2tSd8CSdK6kC83sLyR9QNKHzOxeSbvN7AR3f93MTpD0ZoB9AQAA5F7dc7DcfY2797v7fEkrJW129y9LeljSquJqqyQ9VO++AAAAWkEjvwfrRkmfMbMXJH2m+DMAAEDbCzFEeJi7j0gaKf7/t5KWhNw+AABAK+Cb3AEAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQWN0By8zmmdk/mtnzZvasmV1ZXH6MmT1mZi8U/51Tf3MBAADyL0QP1kFJ33T30yV9UtLlZrZQ0rWSNrn7Akmbij8DAAC0vboDlru/7u6/LP7/XUnPS5oraZmke4qr3SPponr3BQAA0ArM3cNtzGy+pP8j6UxJr7h7b8lj77h7ZJjQzC6TdJkk9fX1fWJ4eDhYe/Jq3759mjVrVtbNyBVqEkVNoqhJFDWJoiZR1CQqRE2Ghoa2uvtA3GPT69pyCTObJenvJX3D3f/NzKr6PXdfJ2mdJA0MDPjg4GCoJuXWyMiIOuF5pkFNoqhJFDWJoiZR1CSKmkQ1uiZB7iI0s24VwtV97v5gcfFuMzuh+PgJkt4MsS8AAIC8C3EXoUm6U9Lz7n5LyUMPS1pV/P8qSQ/Vuy8AAIBWEGKI8FxJX5H0jJk9VVz2bUk3StpgZqslvSLprwLsCwAAIPfqDlju/n8lJU24WlLv9gEAAFoN3+QOAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABDY9KwbADTKxm2juvnRHXptz5hO7O3R1eefposWz826WcgBjg0AjUbAQlvauG1Uax58RmMHDkmSRveMac2Dz0gSF9IOx7EBoBkIWGhLNz+64/AFdMLYgUO6+dEdXEQ7XNpjg96u2lE7dDICFpou7qTbG3ibo3vGYtd7LWE5WluaC3nSMRC3nN6u2lE7dDomuaOpJk66o3vG5Dpy0t0zdiDoNi1h3RN7e2reD/Ip7vW/+v6ntfiGn+vka3+mc2/crI3bRg+vn3QMxC0v19uF8qgdOh0BC02VdNLdvff3QbfpUiRk9XR36erzT6t5P8inuNf/wLjrnfcPTArxEyHr6vNPU09316T1k46NNL1dOGLjttHc9CJv3Daqc2/cHBu2gUZiiHAK5gyks3HbqP7LT57VO+8XeqB6e7r1nQvPSD08s//QeM1tSNqmS5rb28Nr2eaquWCXzrGaOAaqeZ8nDTfTEzpZ6Xmzd2a39v3+YOK6IWpX7XmaYUpkiYBVgjdjOhu3jerqB57WgUN+eNmesQO6+v6nJcXXLOmCNaOr9s7UpG329nTXvM2844PAEeXm3JUqDWKlQaucq88/bdI5QaIndKqp582JD1txQtQuzXk6Lze78H7tTAwRlmDOQDo3P7pjUriacGDcE2uWNDzTN/sDNbcjbpvd00zv7T8YmevVjOGBckMSIYYrkuaxtdrQR5paTKz7zOjeyLpxr3+cWnpOLlo8V9+7+GOa29sjU6FH9HsXf6ymi2Oo1z5vw11x580ktdau0v6SztN5GOJtl/cr0qMHq0Qe3oytpFxdkh5LGp7p3ftCzZ/y4rb5/v6DkU/SzfjkGvfp+qofPaUnf/M7DZx0TJAe0rx8Kq9Hml6ISevOi6479fWf3dOt9/YfnBT+6+k5qba3q5xqn2+590DSNr7355XDZS3trfeuzKnm9vYEOT7TnKdDD/HWco6qFAjp2WpfHRWwKr05mG+RTrmhmXI1i7tgbXzkOa3ZVHv4mLrNk6/9Wex6U0/Cobvukybc37flFf306deDBKN2+CCQJiRWs+7U1z9vQzJJz+GbG44Mp1cKYck3iBQ+SIR6zmmnSlQzRBtyWDXNeTrkEG+tU0iS3pcTv9+oKSmR4+FPq+tlLLsNAmAqHTNEWE03bZq7i1CoV3dX9AsRuqdZ6prt3vv7oMOz1dyK34iu+6QLjUuJX0WRNhil+ZqBtENKzRqCShMSawmUFy2eq3++9lN6+cbP65+v/VRmQ3qV2nrI/fAxV6mno9wNIiGP5bRTJWKH6LtMvT3ddQ+rVru/pPN0yCHeWqeQJL1fu8xit/eNHz1V9/EWdzyMvjOWapsMbdavY3qwqv0UPLEuib3yp5eJ/6e5izBJ4S7CaN6vtVemmk+u5XoVrvrRUzW9/l1mOuTReWnlpO0hrfZTedpP3JXWD/lpNk0vRBY9y6FveCnXyzNxHqoUJMvdIBJy2DhtoG32eTPt/kIM8Uq19xwnvV/LzVur93iLOx7G3VMdD+0wFSFrHROwqn1zhHoztrpqLzCh6pV0F2GtF9FqTsLlehWk2k5y5cLVnJnd+v2B8bqHK6q9wKQ9QVb6hB4ycKQZusniTr5qapcmcMY9h1Ll/gLBxHsgqQ59s2cEHTZOasc0M23cNhr7HJt93sziPF1r0E96v9786I6yQ6v1hJkQx0M7TEXIWscELOZXpdPsTy99sz+gnu5DQS+ilU7C1cwdSfucy/Vg/ee/PENSmE/61Vxg0p4gyy0PfTyk6YUoXVd6V3Ob0LNcqUZpe7gmln1zw9Oxx8fE8y8XJMvdIHJib1ew81tSGJwYzkx6ju2unqCf9H4tF7ql2sNMiOsd18z6dUzA4vts0mn2p5fenm597+KFTR2erdSrMCHNcy7XgzV1KLrR0p4gy63fiOMhTS/ExLojIyP6+n8YrHmf1apUu1oC58TypPNQNaEzrmYjIy8EPb+VC4OdMESU1DMZeii0dHu13CxUTtzxMM3SzY3lmlm/jglYzK9KJ4tPL1kMM0jSN370VNn10jznuQl1m5vBp760J8hy6yddBNr102yl2tUaOCudh2p9DzTi4n9VwvuinYeIKvVMhj5HTWxv6n6l+r9aRJp8PMydcyhV27lm1q9jApbE/Ko0OuXTy0WL55YNWGmfc57qVstk4HLr5+V5NUOlWtTzAaRR56HQ2+3EIaKsJnY3IsxMPR5GRkbq3gbS6aiAhep10qeXcvOm0t7Snbe6pT1BJq2ft+fVDOVql6cg3Sid8BynynJiN2Gm/RCwkKhT3vBfOmee7t3ySmT5lz/5xw2bgN6K2vV51aITAmcnPMepOrHXDo1DwELH+68XfUyS9He/eFWH3NVlpi+dM+/wciBOJwTOTniOpTqx1w6NQ8ACVAhZBCqgs3Virx0ah4AFAEBRp/XaoXE65m8RAgAANAsBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwBoesMzsAjPbYWYvmtm1jd4fAABA1hoasMysS9L3JX1O0kJJXzKzhY3cJwAAQNYa3YN1tqQX3f0ld98vaVjSsgbvEwAAIFPm7o3buNkXJF3g7pcWf/6KpHPc/Wsl61wm6TJJ6uvr+8Tw8HDD2pMX+/bt06xZs7JuRq5QkyhqEkVNoqhJFDWJoiZRIWoyNDS01d0H4h6bXteWK7OYZZMSnbuvk7ROkgYGBnxwcLDBTcreyMiIOuF5pkFNoqhJFDWJoiZR1CSKmkQ1uiaNHiLcJWleyc/9kl5r8D4BAAAy1eiA9S+SFpjZyWY2Q9JKSQ83eJ8AAACZaugQobsfNLOvSXpUUpeku9z92UbuEwAAIGuNnoMld/8HSf/Q6P0AAADkBd/kDgAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELQBjbN0i3nil9p7fw7/YNWbcIADIzPesGAEhp+wZp0w3S3l3S7H5pyVpp0Yrs2/STK6QDY4Wf975a+FnKvm2tKo+vM4Cq0YMFtJKJILP3VUl+JMhk3Vu06YYj4WrCgbHCcqSX19c5JHo80ebowQJaSbkgk2Xvxt5d6ZY32tTenwWflV74eev0BuX1dQ6FHk90AAIW0EoSg8yrhV6ArALE7P5ib0vM8maLu3g/eeeRx1vhYp63wBpaUoB85BqGRdE2GCJEenTt16/WGiYGFst2OGnJWqm7Z/Ky7p7C8maLu3hPNXX4slnHdLX7SXqdqw2sWb9HK+0/KSiO/a45x3HW9el0HVJ/Ahaiyh38nTA3pNHqqWFckJEVtlOq2fOfFq2Q/vJ2afa8Qntmzyv8nEXvQ7W9PBPrxb0eD14m/fRvwrYrzeteT2Bt5ns07lxRzf6rDYqNOI45h2Wrg+pPwMJklQ7+Rk1mHnunIz7RSKqvhnFBZmq4mtDs4aRFKwoBYHZ/Yd+bbsjmdaz24j2xXmyPl0tP3lV/+0sDyI+/Wv3rXk9gDfEeraaHIelc8cg1lfcf+0EhQejjmBsystVB9WcOFiarNLm2EXNDtm+Q9r5xZA5PK8yRqUe9NVy0YnJdbj2zvvlPob4OIC8Tl5esndyOOKW9QYl19/omlU+thx+KXy9p/1Nf52rVe3xV+zomnSuS6l66/4ntlB53+98rDBFOFXoeX7vPb8u7Dqo/PViYrNLBX+/ckDibbpB8fPKyaj7RtMo4/tR29syJX6/WGjZiOOmnf5O+tnn6ZDq9pB49x0gDq5N7g8rVvZ6TfjVzwSrtvxb1vkerfR3T1mbq/hetkK76lfSdPYV/P3dTc+bxhX7/IZ1GXENyih4sTFbpbrC43oF6T4J7d0kfTlguxfewSPnoLakkrjega4Y0rVsaP3BkvXpqGNcbUG0vVNLF9Mm7dHjosZrabt8Qf9xIzf1kun2D9NDl0qH9R5bt3yf98SelpbfE/86StYU5V3FDrfWc9Kt53o0IELW8R0vfY9UOOffMie9x6j5a0nj6c0Q9x3G1tm+Q/vBudHnXjGxuyOhEjbiG5BQBC5NVOvgbcRJM+kTZMyd5uGJ6T2t8T1BcgDm0v9CrMuPocDUMPZyUNGk+bh8Tr1GSZn4yfeSayeFKKvz8yDXJ9Vm0Qnply+RQKR057msdQk36sGJdhR7bRn0NQbXv0cPP61XF3igxVbWv4/SjCr1RtdSs1uO4WptumPzBZsKMWfk6b7SzZgXpHHzdBwELk1Vz8Df6JFiqnnkeeZB4O/o70jUvN7ctcSedpF6IOEnPpdxQWLM/mSY9l0rPcekthV6ukD2lSR9Wqp2sXs9FotJ7dOoHl0rhKu51HHsnft2xd5p7jkij3PsRzdPI4yMvc0FFwEKcZp8cy52o05748jaOn5cv4Ew66ViKaZhJbS4XarP6qoa0ksLMrWfW3lNazyf1Rl8kqp0fJktud16O7TRasc1IJ0d/BYGAhexVmvQYd0LsOUY6OJb/cfy8zDdIOulUq1ybEy9a89Kd0LZvKAzlTfQ29RxTGGpKs42eY+J7q3qOKb/fpDAT+o7PajX6IlFN+2fPK0w+T5KXYzuNVmwz0snRXYrcRYjsLVkb7UmZOOkl3SH3uZvy88WW5eTlCzjTnlx6jqm+zSG+xX1icnppOBr7nbTxP6W7O/RzNxVuICg1rbuwPEm5MNPoO56S7oRt9EWiUvurnZTeyGO70l3CtdxFnJf3IxonR3cp0oOF7C1aIY3+uHCySxpKSRpmaYUTYzW9GLF3Sh4frg1JvUxJPYFpeo5CTFrddEN0crpUmJCcptemlraUCzMXr0vX45Fm3lS5nrNGD2XFfldYcaL77HnZT0qvNERazxBqXueHIYwc9VISsJAPPXOShyPa/YSYdLH4+P8Mt4+kk85Ez069d9zU+xqV65lJ22uTti3lwkyawJb2ol+u56zRF4lm3MlVj0pDpDmaZ4OcydGxTcACspZ0sXj39XC3G1c66WR9UUoKOROPNVI1X01S75+oifv9cj1nzbhI5PmDS6Uh0hzNs0EO5eTYJmABWUu6KBzaH/ZOspycdGItWRv9glCpMH+q0V37ocJM2ot+pWHAPL9ejVapNtwNiBbAJHcga+UuCnn50zONtmiFtOz7k+/26zlGuugHzQkZU/9sSy37TDu5NsTNAe2qUm2oHVoAPVhA1pKGqJK06zBIq/fYpJ03laO5IrlT7ZA2tUOOEbCArCVdLH79dvz6DIPkUy0X/VYPlY1UqTbUDjlHwALyIO5iMfrjQg9IDm43RpW46AMoYg4WkFc9c/hSRABoUfRgAXlGjwgAtCR6sAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAqsrYJnZzWb2azPbbmY/NrPeksfWmNmLZrbDzM6vu6UAAAAtot4erMcknenuiyT9q6Q1kmRmCyWtlHSGpAsk/cDMuurcFwAAQEuoK2C5+8/d/WDxxy2S+ov/XyZp2N3/4O4vS3pR0tn17AsAAKBVmLuH2ZDZTyT9yN3vNbM7JG1x93uLj90p6RF3fyDm9y6TdJkk9fX1fWJ4eDhIe/Js3759mjVrVtbNyBVqEkVNoqhJFDWJoiZR1CQqRE2Ghoa2uvtA3GPTK/2ymT0u6cMxD13n7g8V17lO0kFJ9038Wsz6sUnO3ddJWidJAwMDPjg4WKlJLW9kZESd8DzToCZR1CSKmkRRkyhqEkVNohpdk4oBy90/Xe5xM1slaamkJX6kO2yXpHklq/VLeq3WRgIAALSSeu8ivEDSNZIudPf3Sx56WNJKMzvKzE6WtEDSE/XsCwAAoFVU7MGq4A5JR0l6zMykwryrr7r7s2a2QdJzKgwdXu7uh+rcFwAAQEuoK2C5+6llHvuupO/Ws30AAIBWxDe5AwAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACCwIAHLzL5lZm5mx5UsW2NmL5rZDjM7P8R+AAAAWsH0ejdgZvMkfUbSKyXLFkpaKekMSSdKetzMPuruh+rdHwAAQN6F6MG6VdLfSvKSZcskDbv7H9z9ZUkvSjo7wL4AAAByz9y98lpJv2x2oaQl7n6lmf0/SQPu/raZ3SFpi7vfW1zvTkmPuPsDMdu4TNJlktTX1/eJ4eHhmtvTKvbt26dZs2Zl3YxcoSZR1CSKmkRRkyhqEkVNokLUZGhoaKu7D8Q9VnGI0Mwel/ThmIeuk/RtSZ+N+7WYZbFJzt3XSVonSQMDAz44OFipSS1vZGREnfA806AmUdQkippEUZMoahJFTaIaXZOKAcvdPx233Mw+JulkSU+bmST1S/qlmZ0taZekeSWr90t6re7WAgAAtICa52C5+zPufry7z3f3+SqEqo+7+xuSHpa00syOMrOTJS2Q9ESQFgMAAORc3XcRxnH3Z81sg6TnJB2UdDl3EAIAgE4RLGAVe7FKf/6upO+G2j4AAECr4JvcAQAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwAhYAAAAgRGwAAAAAiNgAQAABEbAAgAACIyABQAAEBgBCwAAIDACFgAAQGAELAAAgMAIWAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBgBCwAAIDACFgAAACBEbAAAAACI2ABAAAERsACAAAIjIAFAAAQGAELAAAgMAIWAABAYAQsAACAwMzds27DYWb2lqTfZN2OJjhO0ttZNyJnqEkUNYmiJlHUJIqaRFGTqBA1Ocnd/yjugVwFrE5hZk+6+0DW7cgTahJFTaKoSRQ1iaImUdQkqtE1YYgQAAAgMAIWAABAYASsbKzLugE5RE2iqEkUNYmiJlHUJIqaRDW0JszBAgAACIweLAAAgMAIWAAAAIERsJrMzL5uZjvM7Fkz+28ly9eY2YvFx87Pso1ZMLNvmZmb2XElyzqyJmZ2s5n92sy2m9mPzay35LGOrIkkmdkFxef9opldm3V7smBm88zsH83s+eI55Mri8mPM7DEze6H475ys29pMZtZlZtvM7KfFnzu6HpJkZr1m9kDxXPK8mf27Tq+LmV1VfN/8ysz+zsw+0MiaELCayMyGJC2TtMjdz5D034vLF0paKekMSRdI+oGZdWXW0CYzs3mSPiPplZJlnVyTxySd6e6LJP2rpDVSZ9ek+Dy/L+lzkhZK+lKxHp3moKRvuvvpkj4p6fJiHa6VtMndF0jaVPy5k1wp6fmSnzu9HpJ0m6T/7e5/IulPVahPx9bFzOZKukLSgLufKalLhfNpw2pCwGquv5Z0o7v/QZLc/c3i8mWSht39D+7+sqQXJZ2dURuzcKukv5VUesdFx9bE3X/u7geLP26R1F/8f8fWRIXn+aK7v+Tu+yUNq1CPjuLur7v7L4v/f1eFi+ZcFWpxT3G1eyRdlEkDM2Bm/ZI+L+mHJYs7th6SZGYfkvTvJd0pSe6+3933qMPrImm6pB4zmy5ppqTX1MCaELCa66OSzjOzX5jZP5nZnxWXz5X0asl6u4rL2p6ZXShp1N2fnvJQx9ZkikskPVL8fyfXpJOfeywzmy9psaRfSOpz99elQgiTdHyGTWu2/6HCB7TxkmWdXA9J+oiktyT9r+LQ6Q/N7Gh1cF3cfVSFUaNXJL0uaa+7/1wNrMn0UBtCgZk9LunDMQ9dp0K956jQtf9nkjaY2UckWcz6bfP9GRVq8m1Jn437tZhlHVETd3+ouM51KgwJ3TfxazHrt01NKujk5x5hZrMk/b2kb7j7v5nFlaf9mdlSSW+6+1YzG8y4OXkyXdLHJX3d3X9hZrepg4YD4xTnVi2TdLKkPZLuN7MvN3KfBKzA3P3TSY+Z2V9LetALXz72hJmNq/DHJndJmleyar8KXZdtIakmZvYxFQ72p4sXiH5JvzSzs9WhNZlgZqskLZW0xI98WV1b16SCTn7uk5hZtwrh6j53f7C4eLeZneDur5vZCZLeTN5CWzlX0oVm9heSPiDpQ2Z2rzq3HhN2Sdrl7r8o/vyACgGrk+vyaUkvu/tbkmRmD0r6czWwJgwRNtdGSZ+SJDP7qKQZKvwl74clrTSzo8zsZEkLJD2RVSObxd2fcffj3X2+u89X4aTwcXd/Qx1aE6lwt5ykayRd6O7vlzzUsTWR9C+SFpjZyWY2Q4XJqQ9n3Kams8InkTslPe/ut5Q89LCkVcX/r5L0ULPblgV3X+Pu/cXzx0pJm939y+rQekwonkNfNbPTiouWSHpOnV2XVyR90sxmFt9HS1SYw9iwmtCD1Vx3SbrLzH4lab+kVcXeiWfNbIMKb4CDki5390MZtjNz7t7JNblD0lGSHiv27G1x9692ck3c/aCZfU3Soyrc/XOXuz+bcbOycK6kr0h6xsyeKi77tqQbVZhysFqFC8lfZdO83KAe0tcl3Vf8QPKSpP+oQqdKR9alOFT6gKRfqnD+3KbCn8qZpQbVhD+VAwAAEBhDhAAAAIERsAAAAAIjYAEAAARGwAIAAAiMgAUAABAYAQsAACAwAhYAAEBg/x9BeGynJxeEqAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import homework\n", "import importlib\n", "importlib.reload(homework)\n", "\n", "N = 100\n", "data_X, data_y = homework.generate_data(N)\n", "\n", "plt.figure(figsize=(10, 7))\n", "plt.grid()\n", "\n", "for cls in range(2):\n", " plt.scatter(data_X[data_y==cls,0], data_X[data_y==cls,1], label='y={}'.format(cls))\n", " \n", "plt.legend(loc=2, framealpha=1)\n", "plt.axis('equal')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## PCA and correlation coefficient\n", "\n", "One possible way to avoid the previous situation is to combine PCA with a filtering method (such as Pearson's Correlation coefficient filter).\n", "\n", "The PCA is then computed normally, but instead of ranking the principal components accordint to the amplitude of the corresponding eigenvalues, they are ranked according to the chosen scoring criterion.\n", "\n", "This is what you are asked to implement in the homework.\n", "\n", "*You can follow these steps:*\n", "\n", "1. Perform PCA analysis on the data\n", "\n", "2. Project the data onto the obtained principal vectors without reducing dimension\n", "\n", "3. Compute the correlation filter on the projected data\n", "\n", "4. Select the important principal components according to this correlation criterion\n", "\n", "5. Build the projection matrix (for dimensionality reduction) using the selected vectors\n", "\n", "## Test cell\n", "\n", "Run this cell to check whether your homework passes the test." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import unittest\n", "import testing\n", "\n", "import importlib\n", "importlib.reload(testing)\n", "\n", "unittest.main(module=testing, argv=['first-arg-is-ignored'], exit=False)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.2" } }, "nbformat": 4, "nbformat_minor": 4 }