{ "cells": [ { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "# Lecture 8\n", "\n", "## The MNIST data set" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "IRdisplay::display_html('')" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "WARNING: Some cells in this notebook take quite a bit of time to run and some\n", "cells crash the kernel when run on the EPFL servers, because of memory\n", "limitations. Therefore this notebook contains the output cells of a run on a local machine.\n", "\n", "Let us load the MNIST dataset and plot some examples of the training set." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAByFBMVEVNTU1OTk5PT09RUVFS\nUlJTU1NUVFRWVlZXV1dYWFhZWVlaWlpbW1tdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVm\nZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4\neHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmK\nioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5uc\nnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2u\nrq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/A\nwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS\n0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk\n5OTl5eXm5ub////2EHiKAAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO3d+Z8kVZm2\n8e5GR1FQcUNcwXHHBXFfx1FRRMcRfcENcBkVFLdxX0BARBEVFbXn333zvrs6TkXkcyLiRD6R\nlV1c35+qa8l6KiKu9MMxljP/B2BnZ056AOA0ICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQk\nICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQk\nICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSLAJ6fyB0DgnPcNFzBJjlhgh\nVTBLjFlihFTBLDFmiRFSBbPEmCVGSBXMEmOWGCFVMEuMWWKEVMEsMWaJEVIFs8SYJUZIFcwS\nY5YYIVUwS4xZYoRUwSwxZokRUgWzxJglRkgVzBJjlhghVTBLjFlihFTBLDFmiRFSBbPEmCVG\nSBXMEmOWGCFVMEuMWWKEVMEsMWaJncqQHpM75ePyYbl142fyT9nfLBmYJXZYsxDSqrNkYJbY\nYc1CSKvOkoFZYoc1CyGtOksGZokd1iyEtOosGZgldliznK6QviovlLMVt8leZkmz2yx+W3lA\n/lueK5+WE5gl12HNQkhrzZKGkGKHNQshrTVLGkKKHdYshLTWLGkIKXZYsxDSWrOkIaTYYc2y\ne0gPybvER+rrZOEwC2fxSpyX6c6Jx3i2fEO8WHfPxo3iL35f1pmleFRukTOd18vjGy0vtGSW\n+8VvGs+TM33PEM/ygQ1/79/lX5I+y8C7ZTDQfdL6QoRUHYaQsmYhpP0ipOxZCkKKEVIFIcUI\nKUZIFYQUI6QYIW0ri2SXiQ9jf/QTaR9m4Sy/lbIw90H5vfS/TQfw4/8l75N1ZnHU/uXXyrnA\ndzZaXrJ1Fm2Np8uZVj+W1FnEldwl47+99XUJqToMIWXMQkj7R0jZsxDSFkKaREhbCGkLIU0i\npC2EtIWQxvxAysk4/ugGuUJeIFrNaxumfZY/yNXiMW6XJ6T2A/7i3yR9Fi98fUWifApdG9Xy\nuq2z/Hxj/JB9vnjprv/5X0jqLGpobsb9n/SnRlfz9hfSn8W79xHxG7ffkY/NQkhpsxDSNkKa\nQkhbCGkbIU0hpC2EtI2QphDSFkLaRkhTdDTc6hU6nxPkq178hU+KDxgd3j5FZuZLLtkwPgvG\n2/w/xH9s62tkzKK1Ot9lZRDNW+XbHV8SpGuDZt6DZdEsoyF9S/xtPn9Kn3qV3C3/kKxZ7puM\n6K5OP5hyEpE/SphlJh8+PuVN28ZXbb1BrhS/+zxF/M9be++HhJQ0CyFtIaRZCKmHkLYQ0iyE\n1ENIWwhpis9nKEsM5RwCz3GH+Ks6kPzBL2X6dVs3jP7L2P/F/CyJGvLh+leZvD5gx1m+uTFo\n6Avi/0z1d/hYea/oi+WNJ3+W0XMI3iH+r2VvMA3V8u7TMEttgtE4ohMgEmap+Y18Wfw+7OPZ\nv1MnhnhQryl8TbwDvdgQrFURUtIshLSFkKYQ0jZC2kJIUwhpGyFtIaQphLSNkLYQ0hhF43MX\nfKyUlQvX9FHpX1DhD0Y33fFh2mZ524ZD/ZgMvugj+Ivib7lpwydjTC9Mtc7iqzNeuVEaepn4\nVgh+D/mU+N4J3XesGZIWD8th6CUm/+n/Lv7c9D0rdpulds3E+K0ZglU+v9BuswR+JC+Xcu2N\n3+J8Yozv8TF2ckw8CyHtOgshbSGkSYS0hZC2ENIkQtpCSFsIaRIhbSGkLYQ06eJq3IXlOJ8d\n9Fopn+t/5A/yQ/L5/jpbw7/tYSlfdEPXy9ltWl77Zuosf5RzDfQ8iLKalzqL9UPybTH9+Ztl\nLyH17wHpf403ZEF5qSFpCddXuTxNvEznaPymNn1vzFGEtPsshLSFkCYR0hZC2kJIkwhpCyFt\nIaRJhLSFkLYQ0iQdCIObQQ4+8kMOtEh2U/epcg3MxDCzZ/Eai7rw5hh80deRuJoXy3dFE/gB\nFVrqe5vPGsqaxU3oj54bktbwZr526yzWD+nj4rNg/E9vF58Z0/KSjbP0a3BI1ST8heAJFdPv\nvq3bRdH4z3+TtFwRNo2Qdp+FkLYQ0iRC2kJIWwhpEiFtIaQthDSJkLYQ0hZCmrS9LnfsI59h\n4TNmfLKQPuUTMR6f97DHhJC8dXzJjafy4zC7L/opFf789MMyWneSLunxUqKvX3m7eJHM126V\nhnzjzOn1q8Wz+B3iORtnajyGT+1a8YkhQRfNppf5GrbLn6Rbx/WZY7oFzQMti6cTsxDSrrMY\nIR1HSJMIKUZIxxHSJEKKEdJxhDSJkGKEdBwhTdJuGKzV+Y4R5f4mvjKp+6ofUDF7mJ1D8sHh\njeWlu/55Hz8Vf/H9kjXLqF9JCekT0voau63aVb1UtJKVf9fX2vVIM42eGNQ6izmkt2z4Zjln\nO760zuuaOyZFSLvPMoqQCClESG0IiZBChNSGkAgpREhtCImQQnpKi3+7l8DulcF3OJ1uJa9l\naSghJF8b5V/uB6j0f6icOnSPZM0yyluohORTdVpfY6WQOv21zbRZajWVJb13d/pfzJ+l4zcN\nnyH1v+IZvcrr99bBhW0NCGn3WUYREiGFCKkNIRFSiJDaEBIhhQipDSERUlV5jkv0hXIfVl2j\n0zbM7Fk+IkpCdzh9pe9e4WWXa8St+AyQ8gP6juvkRVL9C5bMUuP7efoaKG+N98gTrXfwbJzF\nr6/tPjh4nyl+nq0f8buXkDrlZB8ft4Nzf/qzzFywWzxLxHH5br0+Qha9BiHtPksNIR0hpApC\nmoWQjhBSBSHNQkhHCKmCkGYhpCOE1Gpwz1U9TLZtmNmzOBPdL9O/qJx44wleJYPnfL5hw9/7\nPUmdpcbrQGXBzlcrLXqhhll85Vf/0PQNYMoVYX6szX5Dqrpv+0EuLT8+OYvPCZv5Yr6Sy1fP\nTS/phrMQ0q6z1BDSFEKqIKTjCGkKIVUQ0nGENIWQtvUvnrhwaUXD0x2PhmmbRedWeOs/XXwf\nSDf0b+Kd5Kn84ElN5vsC6uYKf8+fpe+H4mdOepO8RPzQikWv1jDL3dIdlbpu4C3lIZu/kzdK\nOXC1qf4g+bNMG9zaIf1eFn4Qh09jGL1IwoeET3vxkdPyVNJjsxBSyix9hDQLIW0jpOMIaRZC\n2kZIxxHSLIS0jZCOI6RZCKnHv9jldBdPnIuur5gxTNssD274jotnp12+8ZCsNEvnESlnd7ih\nll+82yy+akXnbuhyl1vKWp1vHeENcabvMxsrzTJtMEvrj0/O4vW3l4lPMLlqw++0+uAqP5bE\nH/kdz98287Ep4SyElDJLh5DmI6QeQjqOkOYjpB5COo6Q5iOkHkI6jpDmI6QeP3mi3DTST72c\n+fyJ4TDtsyimB7s7OAx5IK9TNSxMLZ7FvHhYTgyavhVl/iw6Oapcr+FzgrwmNThuPyv64pqz\n1AzODmq5emLJLI+KztFyKj4n6kPiW37cJtNPQ52YhZDSZjFCmoWQthHScYQ0CyFtI6TjCGkW\nQtpGSMcR0iyEdJHX6tyQt4SPGn/0W1k4zMJZfKz8QL4v13fPOPSF+HubxRNoefBybw0/7WH5\nLQd3m8W8j3z7wzOBz8veZhmI7h55UrNkIKTEWQhpPkK6iJC2ENJ8hHQRIW0hpPkI6SJC2kJI\n8xHSRT4naPAQzNfJonsgHg2Ts2E8gRvy/SIXPdRwySw3S1mw+7m0vsaOs/heh9/YeLv48pqo\nITeub/vGirPUBLc8OXNf67VISbOkIaTEWQhpFkLqIaQthDQLIfUQ0hZCmoWQeghpCyHNQkg9\nvhCorNW5q52HOaQN0zaLFux83HqT+K4bPqj3PEtwp9XC5wn5hq+LzuBqnaVmsGDX8rDL9FnS\nEFIFIe08Sw0h9RBSDyHNRkg9hNRDSLMRUg8h9RDSbITUc6uUu6ouPzHo+DCHtGHaZuluNOun\nqEw/WnPNWdaUNUu5yeqiS5FSZ8lASBWEFCOkGCFVEFKMkGKEVEFIMUKKEVIFIcUIKZb7DNkd\nHdaGYZYIs8QIqYJZYswSI6QKZokxS4yQKpglxiwxQqpglhizxAipgllizBIjpApmiTFLjJAq\nmCXGLDFCqmCWGLPEHBKAXRESkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQg\nJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQg\nJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQg\nJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQg\nJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkGAT0vkDoXFO\neoaLmCXGLDFCqmCWGLPECKmCWWLMEiOkCmaJMUuMkCqYJcYsMUKqYJYYs8QIqYJZYswSI6QK\nZokxS4yQKpglxiwxQqpglljWLHfJM+W3cqKzZCCkCmaJEVKMkCqYJUZIMUKqYJYYIcUIqYJZ\nYoQUI6QKZollzXKNnJMfyInOkoGQKpglRkgxQqpglhghxQipgllihBQjpApmiRFSjJAqmCVG\nSDFCqmCWGCHFCKmCWWKEFCOkCmaJEVKMkCqYJUZIsdMR0g1yVn4pJzrLCrJmua/jf94tn904\n1/GnfiYrzvJzeZo8Qx6QRS+02yy/l1vlXeLD53Vyhzy24W/7qviL35L6LISUO8sKCGmNWQhp\nGyHNQkjHEdI2QpqFkI4jpG2ENAshHUdI2xySjwRvmBOdxf4u3iE/6dy67Rfi9h/qpM9iXhp7\nbudV4g12WY8/daV8R1acxb/OS3fLX2jJLM7Hx8sVUrbBu8V7xTVdveEfmHlsEVL2LEZIU7MQ\n0ooIiZBaEFIFIRFSC0KqICRCanEJhvTItn9I+Y6/iL/wI3mDaG891i1/TFuyYXzI+k/3ysro\nH/u4rDhL4Ya8f67olH3W+2D40Q2d3WfxPvq63Cn9aC7rh/Qi0eFztf/5Gglecrft4hudvFD8\ny2vvHDO1zqJD0b/ch4on+KDUjovHuh84I4R0ESEREiFVENIIQuohpBGENIKQeghpBCGNIKSe\nEwvpYXmPBAs8N4qXOvwd10mwkz4lM//QJTvJ+6L8yuofqxM9XiFetllnlrJWV/aUbybqkMrn\neh9c+Mh7avDRTrP4Pc0bP4jmAu+8ey76nXiPu6bnyf2y+3YpPiBlguUvtGiW7t3NW+MmGf3+\nj0r5geWnCBFSG0KaQkiENAMhTSEkQpqBkKYQEiHNQEhTnnwhecHCZ5NU98PY58sXvBg08w9d\nspO+KeWorD4jRNvE33avrDPLHf095Y9+3/F+CM4TKqelDH508SxdDdGO8tFxu9R+3OcP+Xs/\nLv0vLg7p1/ICqb10s9ZZ9J5bDhVv8ur36i2kvM95FXV00ZeQEmchpBGEREhzEdIIQiKkuQhp\nxJM0pLeL/+KXiR9U+ONtXmd4r3xI/LnPSLfj3iwz/9AlO8n/x7MH9UfRiRQ+jLsDKfgP6IxZ\ntJFfK/7LXy4tJ1JkzeI1hn5DJaTPyRMy+holpH7PrbN0vCG8euEX9Os/KuVbfiW6vOI28V8x\n/bqts+gvf70M3ucG3+bPaV3qXMNBQ0hJsxBSHSER0myEVEdIhDQbIdUREiHNRkh1T+aQyqLP\nb6Rl3G9LN8NXZOZPtm4Y3QmhrKxUV6K620f621Y6s0Gn45cN/1Np+fGEWXzNRDmXoc/vbtMN\nWQnJ5S2Zpc93hCyzDE52+b5cLt13eKl3uqYls3jRthw0fu8r2+X+3mKdP/ihTL8uISXNQkh1\nhERIsxFSHSER0myEVEdIhDQbIdU9mUN6UJzEX6VlVK+L+GKA52y0/HjDhvFC0MyVle4Wf/62\n/Fl8m8duw/s8oZk/mTXL4JqJvuaTcUpIvr6icZYBPfuiXBniNVzvO19yUm6++HT50sazxD9Q\nu0Xl8lnEv3xwQpZb8U08X9E7pLwrZ777ENLusxDSCEIiJEIiJEI6hpAIqUNIfYR0HCHNtu+Q\nlvFdIn16jIdp3YMNG8Z/Y3fw+reVBzr4yPblPz5jqLt9pG9SkD+LLyjShi8nKXl33du5vW/m\n686fRUfgl85t81pdy2/yoy/9o74WLeGeDd1kPm59JpDfW/24SX/BNflqJX2/H1Dhz79JRl97\nSUjmldvgRhoXPvIBPO9EsuOzENKusxDSxGiE1IqQCCkYjZBaERIhBaMRUitCIqRgNEJq4ENn\ncLcUPdOxbZg5s3h5pds3ZSdFH5WrlfSp0ccJLJrF+qeUnJv+6JaNma89b5bBFUTis2yan/bw\nWfHPJzyN4s+iq5D8R79PfJSUR1/6Riz+tu6H9hKSlSXDctB4xXnmSWTDWQhph1mMkGKEtAQh\nEdIAIS1BSIQ0QEhLEBIhDRDSEl6wK/vRY+n0kLZh5szSXV7UdPDqoRAtW6hhJ82Munyk5aKs\nWb4rV0o/JF8DNPP1rX8Z2WXvkMZZBpyOXsv3V/VanaPxDvSzUYMfKiHdMvl+szgkL+sOHoLp\nX7f8fjWEtNMsRkgxQlqCkAhpgJCWICRCGiCkJQiJkAYIaYlBSL6v5h822oaZM8vXxH+7nl2o\nlcFfliej+HmG5Wkp5cDQzsufxXwWkG5+Uk4C8l7xoOU8IafjL2jnVZ8/0zhL+Qv7BncyHeXD\nx4tkfjUf49Eta6dmGfArapb/FH/Ky3R+fR8vgx/Q6URPFZ+9M72FloSkR6B+NXp3W3Ty1vFZ\nCGmHWYyQYoS0BCER0gAhLUFIhDRASEsQEiENENISfvDLuY7PNWkfJvHaqP5NM8/Ne3LsSrNY\nd7+WV9SeP7NklnJjEZ+f5Rvb6l9zQ/rjRnnUyRtldOlq3nbx25tfUUn4U16wu1YGd1o135q2\nO3q8gjj9W1r3Uf9xw/7IC4rlcy1vb8NZCGn9WYyQCGkGQppCSIQ0AyFNISRCmoGQphASIc3Q\nLdFcMHNTDIdJPHg/2H/QSvvNLCZm8a1CZ+bgBbvyhFlNljVL+Qt9BZEjUEP/lPGX9Yky3W7z\nWlrW3U0dkl/2ixv+lE8Rul6+IP6c4/q6vET0/Z+W6V/QMIt5tbS7OM1ru46mu3FN/RHEc2ch\npGWzENIIQlqMkEYQEiHNRUgjCImQZviZPEVKSM13C7gwTE5I3+z/V2XLfSHnz+I7D0y/tP+f\ndP8/9R7I/13feq5+Q0jTL/Yn8cHb/ejLxH/PbrMU3v3Plus2ftnxEexN8gHx2kM5aHTlRv7T\nSzxLdzQMHhZSzn/xbSFP9jIKQhpBSIQ0FyGNICRCmouQRhASIc1FSCMIiZDm8nMfyuZ4mywc\nZudZvD7Wvy2k71PZdKPKWbOUXRB80Ut6vmag/J/ovlXBoj01NsvHxH+rr1ypPf9Tt8+47zbR\nkX1d2Thjl0y0zjLQLeSW2Af8BT+XwifGzHvIZOss3g3+dcGTLMuq3Q2tF9oMZyGkZbMQ0hRC\nakRIWwjpPCE1I6QthHSekJoR0hZCOk9Izd4jHsbLP/2nC7QNs/MsPselXFqwzsFrZRd0n/Iv\n8rX/vn9EOVbulF1WhOqzuJCrxL/JXd29rWyS4seSOcuA/+APbUQh3Si+i+UTrf00zlL+9OBS\nGldWDpVFYxzNQkjLZiGkKYTUiJCOEFIPITUipCOE1ENIjQjpCCH1EFIjnydT9s8jsnCYnWcZ\nXIW0zsFr5fIin1yjFaGyy8panc+HWTRBwyzlYZI1ZZP4CP6erDPLHs2bpbyp6X4Zn5Tulp7H\nDhXfznOXWQhp2SyEdOIIqToMIS2ahZBGENIShERIA4S0BCER0sClE1JZtbOTCqlcd+IDetFV\nSA2zlF3QfVQ+5ZOUFp2atGgWv1+4Jl+U1O2Ky8Wf+ox4oOXvLfNm2aN5s/gP9ppc8D7nj7y3\nfiK7zEJIy2YhpBNHSNVhCGnRLIQ0gpAaERIhRQipESERUuSSCMnPPLhSTj4kX1VztXjDLH+w\nwLxZfBlN6VYf+P4ZO67+LJql+Jfcc9HMG5qsNMvqWk9X0kldPkC823wfGp/oteOhcjQLIS2b\nhZBOHCFVhyGkRbMUhBQjpPkI6Twh1RDSfIR0npBqDj2kcvCefEi5JmfxIuGtnYQFseWz7BGz\nxAipgpBizBIjpApCijFLjJAqCCnGLDFCqiCkGLPEdgzJ0VzW9yQJaY+YJXZYsxBSiFlizBIj\npApmiTFLjJAqmCXGLDFCqmCWGLPEMq5HSnNYG4ZZIswSI6QKZokxS4yQKpglxiwxQqpglhiz\nxBwSgF0REpCAkIAEhAQkICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBI\nQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBI\nQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBI\nQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBIQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBI\nQAJCAhIQEpCAkIAEhAQkICQgASEBCQgJSEBIQAJCAhIQEpBgE9L5A6FxTnqGi5glxiwxQqpg\nlhizxAipgllizBIjpApmiTFLjJAqmCXGLDFCqmCWGLPECKmCWWLMEiOkCmaJMUuMkCqYJcYs\nMUKqYJYYs8QIqYJZYswSI6QKZokxS4yQKpglxiwxQqpglhizxC7BkB6WL130TjkbOCP/Tx6X\nJzbyZ9kPZokd1iyEtMMs+8EsscOahZB2mGU/mCV2WLMQ0g6z7AezxA5rFkLaYZb9YJbYYc1y\nKYX0R3mJnJOrNi6Tc9PesXGHPDEvqcPaSZf2LA/IvfI68Vvc4G3vh7KPWSIezWP8Vha9BiEt\nnWWPLvVZCGnPCCl2qc9CSHtGSLFLfRZC2jNCil3qsxDSnk1umL/Kx+RGuX/DS3fvFW+OG/v+\nTfpF/VISZtmjvczyY/Ex/hNJmMV7S1v7xTL+Pvcc+fVGy8gJ28UrujeIxyAkQtoVIRESISUg\nJEIipASEREiElICQnkQhNfuN3CXdLvuw/E3WnuXP8nv5vHxUPi6/2mh5ocWzPCY3yYvktfJz\n6X/b3+Va8dLV9yRhlg/KeD99Srnlz0vYR98U//JnivfWohcipPVmISRCOhmElDwLIU0hpHkI\niZBGEdI8hERIo/YY0r/kM+Ilnedu+EQPL57d0efD5EzFW+UfUh9mlaj/KTdvlF32WRn9od1m\n+ZlcI9GWeNXGD2Tmqy2Z5dvyevEf7EL80cvF51p13/sL8RefLl7J2m2W26VfiQ/Zf/Q59BMN\n6VbxaWZeulv+QoS0yiyEREhDhLQAIRHSECEtQEiENERICxASIQ15s0Z3F2nl03W8QFQfZr0V\nxM9tlF3md4HR72+dRbE6Dh+o/iXeP7eIV+2U8s3eEorp3eI3qenXbp3l6xtOwmPoX1/XQuGv\n/D7nz/nEKn/voxvPFn/ex1XCLL7pTJfmneK3s8G3eV2z7BW907b8nVkh+ZcT0jyEREgBQmpF\nSIQUIKRWhERIAUJqRUiEFNhjSP8rQRfPEu+Cp4r/6VaeJ0+R/g98WSaGWS8kHTBrhvSFjbIu\n9xoZnIzj95BXi77DZ6RoqEenX3veLF4H+450u+fX/St8NOMXvC8+Jb8Trev5U64sa5YSkhqq\nftsgJK0IT7926yyj9rhqR0izENJxhLSNkGYhpOMIaRshzUJIxxFSj++leL30k/DR4P9z3Fvi\np+L/k9r/F7l/9PXdDhL/l/X0nRnXDEmXBpRd5v/fP2sWX3agtxA35PtXlss0vF28CvEm6VJ7\nj8z8BfNm8T0W/depoajRL4q/w9eSvEX0L2c/es5J6yze1747xuh1CYOQ3r8x/dqts4zaz2ID\nIc1+aULqIaQeQpr90oTUQ0g9hDT7pQmph5B6CGn2SxNSDyH1eJ3Jyz/Owatzb94YX2zyTfb0\noIirupBKYBPDrBLSn0S3K/AGu0K8YJUwi09N8NM3FYdT8W0RveVcq+98OLiMQm8rD8rMP2De\nLP7t/hNrlyOUkMy75hUbj0jqLDMNQjq1l1EQ0hRC2gkhEdIFhLQTQiKkCwhpJ4RESBcQ0k6e\nLCGZHvlwv+4y8Ko7R0/36Pj0k64h3xbxZB836bm7vTW9YNcwy33SBeLlO58Y9DQ5E/DNKicr\nXjRLCen5Gx+Qu8U3yfQh+9ru3cS8vth6R8TUfeQTmk40pHeJfzkhTSIkQqohpAaEREg1hNSA\nkAiphpAaEBIh1ewxpBZerPLu6kLyKTKzh1klpE9It7dSH335Dekq8RKQ33Oihuxhaf0D5s3i\nZ1e6H51q45TP1rihmVdDLZllmnfD5eK94oeXznu7TZ2lrNp5NfdQHutCSIQ0EyGNISRCmomQ\nxhASIc1ESGMIiZBmIqQx35X+LjvZkHy7j64hP0Vj9BaVrbP4GqBqNYG3S3CrxIRZ+nxfz3vE\nq3neF94IvuvKQ9L6kotnifgSrbJgN/PWK7vN8thF5VPleiTf2HP0QRwTsxDSDrMQ0lKENIaQ\nCGkmQhpDSIQ0EyGNISRCmomQKnwZTv+WJ2cvrsTMXYpJDckN+Yav3k66JaxPlkmdxX+1z4LS\ntVqlF1219WY/DdQHTHlyqOlmMS1/ym7bxdGUkMwPp1j0arvN4vOnfF7VlVIGanm/XTzLExeV\nT71Q9nM90kyEREjTCGkSIRHSNEKaREiENI2QJhESIU0jpEneNf0Fu7OteyshJG+n/lrdBXqO\nS/T0xaxZdOrPg53Bb/qLlJD0LMqW1168Xdz5C2QQki8EWvSSrbNojdSnA/l2rr6Tju8HU2b5\nT5m5lrrbLIGyXQiph5COI6QphFRBSMcR0hRCqoOeq88AABCVSURBVCCk4whpCiFVENJxhDTl\n4EPa2/UlfoCK13x8h9dr5VyFL9bxmtH0qSCpK4gnFdIbxftDlyi930929T99g5pFL9k6y+0b\n3vi+QMoPGY52jW8C2z0LZ6VZ+nztkX/5Ia/aEdJxhERIcxHSCEIipLkIaQQhEdJchDSCkAhp\nLt9hpIR0s7QPMzGL1+Qe7uhaEj/itlZOlY+r0dNSUkPy6pnuf/p8h6THurT8+JJZfGvVq+V5\n4ie3lMe67CUk/85XbvhX+ja03u7VvfKMDb/P5c8SuFcOb9WOkEYQEiHNRUgjCImQ5iKkEYRE\nSDP4xgW+v14JaeaTK4bDhLP4NAEfBG+TlmB8Ib7/WzL44svE+3f+LDvwbQkcki65+LXM/Mkl\ns9wi/jN/3D3k4UOyv5B8DOgOEb51hBdcfLR6Ap9voXeUd18n3U5x9q8Wv0VmzRLwY1b8Ow9l\nsYGQZiEkQhpHSLMQEiGNI6RZCImQxhHSLIRESOPeJ6Whl8qfpH2YcJbPysxy3iG6J8JPfasC\nL8w9dJH37DVSfqD2/IHUkHwihd9qHNKLN9a8pEPKn+lzTryIV65f8J0kWl+yYRb/dU8XPZrE\n//JpJf7lPr3hZ+Lv1YNAf/dl0RuMv8O32rhKEmYZ5f3h3/kWWf5ChERITQipNgshEVIDQqrN\nQkiE1ICQarMQEiE1IKTaLEtD8gkgfs6jzux4xo4LdkfDhLOU0zgC3uafFz/Jcfra/2/JrZ3a\nD6y5aqfnTrb85JJZXiTecg7pLvE//WRSd9X6kg2zeKVQXTxVY/id1ntLt+l8dvXho1pAvU3u\nlqxZRt0kJ7tqR0iNCImQIoTUiJAIKUJIjQiJkCKE1IiQCCkSPObyrM/IWfRggaNhwlm8baOI\nPrfhM08W/bZFsyzj66i8LLS3kMrpQA7YzwnxP32uVeurLZrFyfb3mG/lsfyXL5+lpjz60nvG\n9/1YOAshtcyyDCERUhUhzUdIhFRFSPMREiFVEdJ8hERIVR+RQUjLF+yOhsleKVsudZbBDSJ1\nZK89ix/zWe7H6N3jw9irra2vtmiWP4iuLPIEvihp0ZljGbPU+F3fS3d+COYu/yNASCFC2nkW\nQpqFkOYjJEKqIqT5CImQqghpPkIipIgvr3mDpC3YHQ3z5Ahpb0+j+Kj4MPZ1WrVrr/YyywoO\naxZCChFS8iwrOKxZCClESMmzrOCwZiGkECElz7KCw5qFkEKElDzLCg5rlgUh+XYVgwU7P7Bj\n52EOacMkzuKLnrw2tN+QVsAsMUKqIKQYs8QIqYKQYswSI6QKQooxS4yQKggpxiyxRSF5Eeo1\nUhbsfOuRnYc5pA3DLBFmiRFSBbPEmCVGSBXMEmOWGCFVMEuMWWKEVMEsMWaJZTzWJc1hbRhm\niTBLjJAqmCXGLDFCqmCWGLPECKmCWWLMEnNIAHZFSEACQgISEBKQgJCABIQEJCAkIAEhAQkI\nCUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkICUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkI\nCUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkICUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkI\nCUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkICUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkI\nCUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkICUhASEACQgISEBKQgJCABIQEJCAkIAEhAQkI\nCUhASECCTUjnD4TGOekZLmKWGLPECKmCWWLMEiOkCmaJMUuMkCqYJcYsMUKqYJYYs8QIqYJZ\nYswSI6QKZokxS4yQKpglxiwxQqpglhizxE5bSA/J++Ws3CcnNUsaZokd1iyEtNYsaZgldliz\nENJas6RhlthhzUJIa82ShllihzULIa01SxpmiR3WLKcrpO/Juc7dclKzpGGW2GHNQkhrzZKG\nWWKHNQshrTVLGmaJHdYshLTWLGmYJXZYsxDSWrOkYZbYYc1yukL6hrih78iJzpKGWWKHNQsh\nrTVLGmaJHdYshLTWLGmYJXZYsxDSWrOkYZbYYc1CSGvNkoZZYoc1y6kJ6YtypTikv8sJzOJ1\nw7fKHbJ8goRZVnCpz/JPeVBeKD5Unin/I3+VhbMQUu4shLQ3hFRBSMmzrOBSn4WQJhHSPlzq\nsxDSJELah0t9loMP6V/yD/Fy2Z3ygo0zndvlCakPs3AW//2vknJ20ImGdFnnC3JSs6zgUp/l\n63JZn48Xf/RS+a20z0JIubMQ0t4QUnUYQkqdZQWX+iyENIaQ9uZSn4WQxhDS3lzqsxxySI/J\nJ+TcJH+vowuHWTjLPVJ+yZfEyzPL/6jFs/gkpcFO8nwnMIv5BjD/IWc7fmfzLmvdSrvN8gP5\nvSx/jeWz/HTjJVKi0ad+6uXVd0j5wsMbbbMQUoiQ1piFkMYQ0nGENIKQxhDScYQ0gpDGENJx\nhDSCkLZ5JerbcrOUw/hp8nzRfN+S54m/+B6pHeNLdtLj8lrx618ny0/02G0Wc8UHEtJHxPvj\nbM39G2vOoqNEB+XDN8lT5HK5Xn4sf6nwjWv8HT7QfQ/d/vJn6yzaFdXFOa8nf0b8LTrfrOW1\nCamCkHJmIaQphBQjpB5CmkJIMULqIaQphBQjpB5CGvF9GZyR83TxyscvpP8Drxd/262SuWqn\nA+H+MsanpfU1IosP3j/Jm6SE5PWx/c2iVbEbxb/cvdwij8if5fPiL+h97rvyI0mdxTl0G6Lq\nXHd8T/M75pJZ/I7f/SKv1UXf5nPcfLQSEiER0jZCmkJI2whpCyFNIaRthLTloELyGoH+s7k0\n5Hy+LD+X4If8n4cfknK0+1gLh9k5JG+w1teI7Pb/4Ps/lMth4qN6L7N4N1yx4Up8M4KfyGB9\nx5/ztzx7wx+8TlJn8RNIp+OYDult4reCX8mSWXwE6nQOv9ePf+8bhZAIiZC2EdIUQuohpBgh\nTSGkHkKKEdIUQuohpNihhOQcPibdIXu1TC/1+MT5c321pbuEkB6V1teIpIbkMzv+KCvO4uW4\nl4u6uEqCBdQLvPF9wkB3isNHJWuWX8u/ibbBW8QX0HhGX9fhezTe0afVxsc7vxR/VLvRx7xZ\nfiN+S9Gu8PE4/gNectXK3SelusI3nIWQQoS0yyyEVEFIbQiJkEKE1IaQCClESG0IiZCGvN7j\n2+p1R+vL5C6prb9d8MCGV1zKga57Rr7Af0xCSJrMS0M+Ej4u/rxP9Ph7X8vrLpmlbxCSP/KJ\nVYtebd4s7xZvCS2lVu9y+DfpN+TnMvxOsmb5sPSX3rzU+1n5g0y/RtYsPto8graLEx//ge78\nLu87/2vOLIQUIqRdZiGkbYS0ACER0hAhLUBIhDRESAsQEiEN+ekKpQSt1XkBpvoD3lM/FD3/\n8Vyf2nqgPkzbwatjpby0r6rxAo/vhujP+UaIOmPmits2vFzkyqZfOz+kN8uiV5ucxWuVzxKH\nUSvWJwZ5BfFsT8sZTA2rdu7zsm0e9H3iR3XM/MWLZ/EZbP7Fz9kYX9b14ekVPkIiJELqIaRJ\nhHSEkEYQ0iRCOkJIIwhpEiEdIaQRBxCSF+z+XcrhqqWxBzpe6vHTBXyykK/6eOt2Py8S3+tv\n9DBuPXi1t8ov8W0I/UueKv6cj5P+LP8to0/gXDRLnzeOL9YqIb1aFr3a5CxeoithfHDjFx3/\nwb7GxrO8QryqtmJI3r4+J0pvqH7oqY+NIKvLfE/PGYtpS2fxo1j9m/Q/AuN7vtz8pAvpXpkz\nCyGlzNJHSIR0DCEtRUiEdAwhLUVIhHQMIS1FSIR0jEN6p5yr0K0znu3TfmrfYV4umzNM28Hb\nuz7qQkj+/M/E5ya5YF8XoyWz8tAKP3VgdJsmrNr5N3nd0B/5HJlFrzY5i3eUb+txdox3pZdc\nfdpM9/n/kaxZpgf1A0q0inrbK6VsIV2AVLsL7/JZyhVxWo2rnjpl/qpXG7v3/9mzEFLKLH2E\nNDIoIRHSXIQ0MighEdJchDQyKCER0lyENDLoky8k89/u61d8Y43RYAau2fC9I6YXyY6GyQmp\n8KUvXcY+eekGqf7A4ln6/NJ+xKN/09qrdua/zmtMukTr+s5XxE90KUumDr0Laeb73LxZvMl/\n3ak85fQYH1beQpp25hlc82Yx33zWz//UrviaDK5Q8y1ZfBPXl0i3nujNN3sWQkqZpY+QCGmI\nkBYgJEIaIqQFCImQhghpAUIipDqf8vHgRV4aUyrXRA29R1rvepofUkDnz3xw7ZDM193sL6QW\n3giO6NqNlrvDTM7ie7CUE4C0POpDpfoDPoGo+/6Z77nzZim+Id2u8JupL5nz/WW1Da4NTl7y\nD82ehZBSZgkQEiGFCKkNIRFSiJDaEBIhhQipDSER0lzflEFDL5bRm5zUh1klJO8bXS71CfFZ\nTU/ikLw1/MBYh6R7kKTO4gexeG3MJwDpL/f1YX62i2/P6+8os/ixv95COoWn5SHArdtF19IF\nvVy47uj58lIhJEKagZAIaYiQFiAkQhoipAUI6cBC8ikL5ag0rz0sauhomLZZFIKfjOFf/gHx\ncw79RZ8977tb9h/ZaT6JP/X+EQFf+39wIf1VyqUV6SEVvhej/mN+cNw+V/ymVs4y8Rdu2vjn\n9KkQS2YxvbRP8fCDSz4tXxSf++Dt4jN4CImQZiAkQhoipAUIiZCGCGkBQiKkIUJagJAOJyT/\nP9VlTcp8F4TxJ2HOGKZ9ljuljOEnCbhnbwofK90XfY9Cb53pk/UTDl4vXR16SLrkYs1ZdIcI\nv21dVuMt5KXe0YecJMwyzQ9S7Y6XtlkIKW2WAUI6T0ijCGkWQjpPSKMIaRZCOk9IowhpFkI6\nT0hVOuP+7VIO3pnPd5gzTPuG8bUdN8u5gI+VK0X5rLkiFDjMkPyEihKSlqnWnsXb3ScGfVV8\n685ndbxS5uXV1tfND8krv13ibbMQUtosA4R0hJAChDQbIR0hpAAhzUZIRwgpQEizEdIRQurz\npf36TeVA9WkWLff0mxgme8MslzCLzzw5uJDukRJS6wF8yvbRACGlI6Q9zLIjQqoOc6p2EiGt\njJCqw5yqnURIKyOk6jCnaicR0sou1ZB80/7+qpgvSlowcG2Y07eT3ioHHFJ3I5KTmGVHhFQd\n5vTtJEJaDyFVhzl9O4mQ1kNI1WFO304ipPUQUnWY07eTCGk9l2BIPgHondI1dKu0nHQzZ5jT\nt5O81Oma/ktOdBYbhKSrk05slh0RUnWY07eTCGk9hFQd5vTtJEJaDyFVhzl9O4mQ1kNI1WFO\n304ipPVcgiGZb23SnRiU29DRMKd5Jy23Zki64emJzbKjw5qFkEKndhZCWgUhVZzaWQhpFYRU\ncWpnIaRVEFLFqZ2FkFax4zNkcx3Whjmls/xefPual8sPN05slh0d1iyEFDq1sxDSKgip4tTO\nQkirIKSKUzsLIa2CkCpO7SyEtApCqmCWGLPECKmCWWLMEiOkCmaJMUuMkCqYJcYsMYcEYFeE\nBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASE\nBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASEBCQgJCABIQEJCAlIQEhAAkICEhASkICQgASE\nBCQgJCABIQEJCAlI8P8BeE2obAWwvz8AAAAASUVORK5CYII=", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "library(keras)\n", "mnist <- dataset_mnist() # loading the data set\n", "par(mfrow = c(4, 6), oma = c(0, 0, 0, 0),\n", " mar = c(.1, .1, .1, .1), pty=\"s\") # setting some parameters for plotting\n", "for (i in sample(1:60000, 24)) {\n", " image(t(mnist$train$x[i,28:1,]), col = gray.colors(256, rev = T), axes = FALSE)\n", "}" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "The inputs are loaded as 60'000 arrays of 28x28 pixels with values between 0 and 255." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[1] 60000 28 28\n", "[1] 0\n", "[1] 255\n" ] } ], "source": [ "print(dim(mnist$train$x))\n", "print(min(mnist$train$x))\n", "print(max(mnist$train$x))" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "To prepare for fitting with Linear Discriminant Analysis we transform the 28x28-\n", "dimensional arrays into 784-dimensional vectors, i.e. the training data is\n", "reshaped from 60'000x28x28 to 60'000x784. We also search for all pixels\n", "that have zero variance accross the whole dataset and exclude them. These are\n", "usually pixels at the borders of the images that are 0 for every input." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "X <- array_reshape(mnist$train$x, c(nrow(mnist$train$x), 28^2))\n", "idx.zero.var <- apply(X, 2, var) == 0 # identify pixels with 0 variance\n", "data.train <- data.frame(X[, !idx.zero.var]) # exclude those pixels\n", "data.train$Y <- mnist$train$y" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "Now we load the MASS library and perform LDA. This will take quite of time,\n", "because the data is rather high-dimensional. If you have installed R on a\n", "sufficiently powerful personal computer, I recommend you to run it there.\n", "(This cell does not run on the EPFL servers, because of memory limitations)." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Warning message in lda.default(x, grouping, ...):\n", "“variables are collinear”\n" ] } ], "source": [ "library(MASS)\n", "lda.fit <- lda(Y ~ ., data.train)" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "Let us prepare the test data in the similar way as the training data and plot\n", "the mis-classification table and the accuracy." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ " \n", " 0 1 2 3 4 5 6 7 8 9\n", " 0 940 0 15 5 0 8 12 2 7 9\n", " 1 0 1096 32 5 12 8 8 30 27 7\n", " 2 1 4 816 25 6 4 11 15 8 1\n", " 3 4 3 34 883 0 44 0 9 27 13\n", " 4 2 2 21 4 888 12 25 22 20 63\n", " 5 13 2 5 25 4 735 29 2 53 6\n", " 6 9 3 37 3 7 15 857 0 10 0\n", " 7 1 0 9 16 2 10 0 864 6 37\n", " 8 9 25 57 29 10 38 16 4 790 12\n", " 9 1 0 6 15 53 18 0 80 26 861" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "0.873" ], "text/latex": [ "0.873" ], "text/markdown": [ "0.873" ], "text/plain": [ "[1] 0.873" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "X.test <- array_reshape(mnist$test$x, c(nrow(mnist$test$x), 28^2))\n", "data.test <- data.frame(X.test[, !idx.zero.var])\n", "data.test$Y <- mnist$test$y\n", "p <- predict(lda.fit, data.test)\n", "table(p$class, data.test$Y)\n", "mean(data.test$Y == p$class) # accuracy" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "Next, we repeat the same with multinomial regression.\n", "We set the argument `MaxNWts = 8000` to tell the function that it is allowed to\n", "build a model with at most 8000 weights. This cell will take even longer to\n", "evaluate, than the one before." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# weights: 7190 (6462 variable)\n", "initial value 138155.105580 \n", "iter 10 value 33282.974291\n", "iter 20 value 26173.678591\n", "iter 30 value 24672.543031\n", "iter 40 value 24229.953056\n", "iter 50 value 24010.800008\n", "iter 60 value 23932.917353\n", "iter 70 value 23882.106974\n", "iter 80 value 23818.241965\n", "iter 90 value 23749.036339\n", "iter 100 value 23697.758066\n", "final value 23697.758066 \n", "stopped after 100 iterations\n" ] }, { "data": { "text/html": [ "0.8976" ], "text/latex": [ "0.8976" ], "text/markdown": [ "0.8976" ], "text/plain": [ "[1] 0.8976" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ " \n", "p2 0 1 2 3 4 5 6 7 8 9\n", " 0 920 0 7 9 0 11 5 4 5 3\n", " 1 0 1118 15 3 9 10 7 26 33 9\n", " 2 1 3 915 26 2 2 6 23 9 0\n", " 3 1 3 10 888 2 34 0 2 13 10\n", " 4 3 1 10 5 917 12 8 11 11 70\n", " 5 30 1 6 19 1 668 12 1 21 6\n", " 6 16 4 17 7 11 35 919 2 14 6\n", " 7 2 1 8 10 3 10 1 916 10 22\n", " 8 6 4 39 35 4 102 0 5 847 15\n", " 9 1 0 5 8 33 8 0 38 11 868" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "library(nnet)\n", "mn.fit <- multinom(Y ~ ., data.train, MaxNWts = 8000)\n", "p2 <- predict(mn.fit, data.test)\n", "mean(data.test$Y == p2)\n", "table(p2, data.test$Y)" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "Now we prepare our first feature representation. To have a similar situation as\n", "in the XOR-Problem, where all input data was around 0, we scale the data first\n", "to lie between 0 and 1 and then we subtract for each pixel the mean over the\n", "entire training set, such that each transformed pixel has a value between -1\n", "and +1. When we do feature engineering there are always many choices.\n", "The exact scaling, for example, does not matter here. What matters is, that the\n", "784 dimensional vectors point in different directions, such that the pair-wise\n", "inner products are sometimes positive and sometimes negative and the relu\n", "non-linearity has an effect. If all inner products would be positive, the relu\n", "non-linearity would not have any effect and the feature representation would be\n", "a linear transformation of the original data. Linear feature transformation can\n", "be sometimes useful (we will discuss this towards the end of this course), but,\n", "for example, the XOR-problem could not be solved with a purely linear\n", "transformation." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "X.p <- X/255\n", "m.X.p <- colMeans(X.p)\n", "X.p <- sweep(X.p, 2, m.X.p) # subtract the means from all input points\n", "feature_vectors <- t(X.p[1:200,]) # we just pick the first 200 images as feature vectors\n", "relu <- function(x) ifelse(x > 0, x, 0)\n", "newdata <- data.frame(relu((X.p %*% feature_vectors))) # create the feature representation\n", "newdata$Y <- data.train$Y" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "Let us run now multinomial regression on the feature representation.\n", "Note that we have less parameters now, because we went from 784 input dimensions\n", "to a feature representation with 200 features." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# weights: 2020 (1809 variable)\n", "initial value 138155.105580 \n", "iter 10 value 42283.013913\n", "iter 20 value 34453.953276\n", "iter 30 value 30854.088830\n", "iter 40 value 29663.308584\n", "iter 50 value 29000.061505\n", "iter 60 value 28470.259300\n", "iter 70 value 28123.439179\n", "iter 80 value 27927.197102\n", "iter 90 value 27807.099123\n", "iter 100 value 27725.248837\n", "final value 27725.248837 \n", "stopped after 100 iterations\n" ] } ], "source": [ "mn.features.fit <- multinom(Y ~ ., newdata, MaxNWts = 3000)" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "To test the fit, we also need to transform the test data in the exact same way\n", "as the training data. Note that we subtract the pixel-wise means of the\n", "training data, because it would not be the same transformation, if we subtracted\n", "instead the means of the test data. Next we compute the feature representation\n", "of the test data and the predictions." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "0.9008" ], "text/latex": [ "0.9008" ], "text/markdown": [ "0.9008" ], "text/plain": [ "[1] 0.9008" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ " \n", "p3 0 1 2 3 4 5 6 7 8 9\n", " 0 915 0 3 1 4 11 6 2 2 3\n", " 1 0 1119 16 7 6 0 3 17 5 8\n", " 2 6 3 894 13 1 2 3 5 5 2\n", " 3 2 2 28 807 0 10 1 8 6 8\n", " 4 17 0 20 0 913 11 35 11 13 38\n", " 5 14 2 4 19 3 751 26 1 14 2\n", " 6 7 3 11 10 6 18 870 4 5 4\n", " 7 2 0 17 11 9 4 1 934 4 28\n", " 8 12 6 26 134 8 80 12 5 916 27\n", " 9 5 0 13 8 32 5 1 41 4 889" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "X.p.test <- sweep(X.test/255, 2, m.X.p)\n", "p3 <- predict(mn.features.fit, data.frame(relu(X.p.test %*% feature_vectors)))\n", "mean(data.test$Y == p3)\n", "table(p3, data.test$Y)" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "Let us turn now to learning the features with a neural network.\n", "We use `keras` here, as it is much faster than `ADtools`." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "lines_to_next_cell": 0 }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "`geom_smooth()` using formula 'y ~ x'\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAIAAAByhViMAAAACXBIWXMAABJ0AAASdAHeZh94\nAAAgAElEQVR4nOzdd3gc5bn38WfK9qJdSZZsyw03XAFTbLBxw5hejQEbCBAgBQ4HOIQQEl4I\nOYSEZk4SkpBGSei9V9sQAwZjMGAcF9zlpmKV1fY2M+8fu5YL7tJqVqPv58qVa3a13r0H7c7+\n9Mw89yMZhiEAAADQ+clmFwAAAID2QbADAACwCIIdAACARRDsAAAALIJgBwAAYBEEOwAAAIsg\n2AEAAFgEwQ4AAMAiVLMLMEdzc3OhX8Ln80WjUQv3f3a5XHa7PRqNappmdi2F4nA4DMNIp9Nm\nF1IoiqJ4vd5UKpVMJs2upVAkSfJ4PNFo1OxCCsjr9cqyHA6HzS6kgNxudyqVsvbRxul0xuPx\nTCZjdi2FYrPZFEWx9tHG7/dnMpl4PF7QF5JluaSkZE8/7aLBrgOODpIkaZpm4WAnhJBlWdM0\nCx9qDcMwDMPCOyjLsizLokM+EWaRZTn3YTS7kALK/R6tvY+SJOm6buF9NAxDlmVr76OqqsLq\nR5tiOOBwKhYAAMAiCHYAAAAWQbADAACwCIIdAACARRDsAAAALIJgBwAAYBEEOwAAAIsg2AEA\nAFgEwQ4AAMAiCHYAAAAWQbADAACwiC66VmyhNTc3r1mzxufzVVRUmF0LAADoKhixa2fxePyG\nG24YMmTIpEmThg8fPnPmzJqaGrOLAgAAXQLBrp39/Oc///rrr99///1oNLp48WIhxJVXXpnN\nZs2uCwAAWB/Brj3V1tY+/fTTf//734cPHy6EqKqq+vvf/75q1aqPPvrI7NIAAID1EezaU3V1\ntdfrHTRoUOs9Xq938ODB69evN68oAADQVRDs2lNFRUUsFmtoaGi9R9O0jRs3VlZWmlgVAADo\nIgh27emQQw4ZN27cDTfcEIlEhBCZTOaOO+6w2+2TJk0yuzQAAGB9tDtpZ3/6058uu+yyI444\nYtiwYWvXrnW5XA8//LDb7Ta7LgAAYH0Eu3bWo0ePd95559NPP62pqQkGg8cff7zD4TC7KAAA\n0CUQ7NqfLMvjxo0LBoOhUMgwDLPLAQAAXQXX2AEAAFgEwQ4AAMAiCHYAAAAWQbADAACwCIId\nAACARRDsAAAALIJgBwAAYBEEOwAAAIsg2AEAAFgEwQ4AAMAiCHYAAAAWQbADAACwCIIdAACA\nRRDsAAAALIJgV1i6YXYFAACgyyDYFUQmK+Z+Kd3yiP/5D11m1wIAALoK1ewCLMgwxNUPBjc1\nyELIWxqV88cnZPIzAAAoPBJH+5MkMWZIOrdd1ywvXGk3tx4AANBFEOwK4owxSVnKb7/xmdPU\nWgAAQFdBsCuIHqXaMUPy8yYWrbJvblTMrQcAAHQFBLtCOW9CPtgZhnjnCwbtAABAwRHsCmX8\nSKMyqOe23/3CmclKe388AABAGxHsCkWWxSlHJ3Pb4bg0bwlTKAAAQGER7Aro1GNStm39ZN5c\nSEM7AABQWAS7Agp69bHDUrnt5RvUVZvpGggAAAqIYFdYZ4xJtm6/9TlTKAAAQAER7AprZL9M\nv8psbvv9rx2RBFMoAABAoRDsCu600flBu1RGmvsVg3YAAKBQCHYFN3VUyu3I97R7fYHTMMwt\nBwAAWBbBruBcDuOEI/JTKDY3KovX2sytBwAAWBXBriOceWyidZulYwEAQIEQ7DpC3wpteN9M\nbvvT5Y6GMP/ZAQBA+yNhdJDWvieaLt6h7wkAACgAgl0HOX54KuDNLx371ufOrGZuOQAAwIII\ndh3EpopTjsoP2jVF5AUrHObWAwAArIdg13FOPSYpb/vvzRQKAADQ7gh2HacyqI8enM5tL15r\nq65TzK0HAABYTBddlt5mK3gzOUmSbDabsXM/4nPGZRassAshDEO8sdBzw7TEHv51JyDLshDC\nZrPJsmX/PFAUxTCMDni3mEVRFCGELMsW3kdJknIfRrMLKSBJkkSHHNZMJMuyqqq5PbWk3IdR\nVVXDul3sFUWx/NFGbPv2L+gL7f1rt4sGO4ej4Je4SZJkt9t3uXPcSNGvu7G+VhJCzP7Sfs3Z\nhs/dWT/DucOQzWZTVcu+i3IHWQt/l+SODoqidMAnwiy5YGfhHRTbvk6svY+5QGDto43oAn8q\nW/vDmPskmn5EteyHZO+i0WihX8Jms8Vise/+7XX6Mdk/ve4VQiTT4sV52enjO+ugndfrVRQl\nHo9rmmWn+LpcLsMwksmk2YUUis1ms9vtmUwmFouZXUuhyLLs9/s74CNvolwasPY++v3+RCKR\nzWbNLqRQXC6XqqqJRCKdTptdS6E4HA5VVa19tHE4HNlsttAfRkVRnM49Xqlv2b8MitZJR6Va\nR+le/8yl6+aWAwAArINg19EcNmPqqPwIUF2z/OnyXU/XAgAAHByCnQnOHru978lrC1ym1gIA\nAKyDYGeCyoA2+tDtfU/W1XbRKx0BAED7ItiZ4+xjt8+ZeG0BzYoBAEA7INiZY9TATL/K/PSu\nuV85wnHLNtQAAAAdhmBnmrOOzU+hSGeld75g0A4AALQVwc40U0al/Nv6nrzxmUuj7wkAAGgb\ngp1pHDbj5KPyg3b1IfnT5ZZtxg0AADoGwc5MZ4xJtPY9efUTzsYCAIA2IdiZqTKoHzc03/dk\nyXrbqs30PQEAAAePYGeys3boe/L6ZwzaAQCAg0ewM9nh/TP9u+f7nnzwtSMU4zcCAAAOEjHC\nfGcdl59CkdHoewIAAA4ewc58Jxyxve/J6wucWc3ccgAAQGdFsDOfXTVOPSY/aNcYlucvo+8J\nAAA4GAS7onD66ISy7VfxyicuU2sBAACdFcGuKFQE9HHDU7nt5RvUZRts5tYDAAA6I4JdsZh+\n/Pa+Jy99zKAdAAA4YAS7YjG4V3ZYn0xu+5Nl9i2Nirn1AACATodgV0TO2zZopxvilU8ZtAMA\nAAeGYFdEjhuW7lmWb3by7heOcFwytx4AANC5EOyKiCyJc8bmB+1SGemthTQrBgAAB4BgV1xO\nPmqHZsWfuTJZc8sBAACdCcGuuDhsxmmjtzcrnreEZsUAAGB/EeyKzjnHJWxqftDuhY9chmFu\nOQAAoNMg2BWdgFefdFg6t72+Tv16Dc2KAQDAfiHYFaPzjo9L22bEvkizYgAAsH8IdsWoX6U2\nakC+WfEXq+zralVz6wEAAJ0Cwa5ITdthhbFXPqHvCQAA2DeCXZE6elD6kO75Zifvf+1ojvKb\nAgAA+0BcKF7njs33Pclo0usLGLQDAAD7QLArXpOPSJX69Nz265+5kmlWGAMAAHtDsCteNsU4\n89j8oF0kLs39mmbFAABgbwh2Re2M0QmnPd+h+KWPXTrNigEAwJ4R7Iqaz22cOCqV297cqHy6\nzG5uPQAAoJgR7IrduWMT8raL657/yG1qLQAAoKgR7IpdVbk2dlh+0G7FRvWbdawwBgAAdo9g\n1wlcMHF7s+Ln5rHCGAAA2D2CXScwuCq74wpjq7ewwhgAANgNgl3ncP6EeOv28x8xaAcAAHaD\nYNc5HDkwM7gqv8LYR/9xbGlUzK0HAAAUIYJdp3H+hPyVdrouXprPoB0AANgVwa7TGDc8VVWm\n5bbf/cLRFOF3BwAAdkI46DRkSZw3Pj9ol9Gk1xYwaAcAAHZCsOtMph6ZKvPrue3XPnVGE9Le\nHw8AALoUgl1nYlOMc47LD9rFU9LbnzvNrQcAABQVgl0nc8aYpNdl5LZfnO/KZBm0AwAAeQS7\nTsblMM4Yk8xth6Ly7K8c5tYDAACKB8Gu8zlnbMKu5gftnpvn0nVzywEAAMWCYNf5BDz6SUel\nctu1zcrHSxm0AwAAQhDsOqnzxyeUbb+6Z/7tMgxTqwEAAMWBYNcpVQa18SPyg3Zra9Wv1tjM\nrQcAABQDgl1ndcGEhLRtRuyz89ym1gIAAIoCwa6z6t8je9TAdG578Vrbsg0M2gEA0NUR7Dqx\ni09ItG4/+T4rjAEA0NWpZhewP4yWle+9/OYX6yNqxaHjzzpnbK/dTAPVG//zzqtzvt7QnHFV\nDB57xtnj+1r/7OTQPpmRh2SWrLMJIRatsn+7ST20V9bsogAAgGk6wYhdesXjv7jl4UWZ3iOG\nVjR98MBN//t6zXcmgUYW/uGmW//1dbr7kKH9bGvfuP+nd75Va0atHe6iyfHW7ac/sH6WBQAA\ne1H8I3bNcx5/pfGo6/5+8ySfEGJ8yY0/fOblb0675nBlh8c0zH3+g+ykW+/9n9FOIcT0Mb6r\nb3765W9Ou/owk2ruOKMGZIb3zSyttgkhFqywr9ysDq5i0A4AgC6q6Efskl9/sVSMmjjWl7vZ\n/Zije0cWLlyx02OMtavXSiNHH+XM3VQPHTJQbtm4MdzBpZpk5uTtV9o9x/RYAAC6sKIfsdtY\nXa1XTult33a7Z88q0Vy/VRNihyG7kVf+8a9y2bY70sv+s0p3H1bp2/ZjwzC2bNnS+mifz6co\nOw74FYqiKEbhewePGaIN7pVduUkVQsxfZt+w1X5Id63QLyqEkCRJCNEx/yXNIkmSJEkW3kdZ\nloUQ1t5Hy/8SW1l7HyVJkmXZwvuY+zBafh+t/WHMfS12wD7m3i17UvTBLhaLCbd7+ziU4nE7\njbpIVIiS1vskV7BHfk6o3rL0+Vn3vtM86LJzj9rW5U2k0+mzzz679eGXX375tdde2wG1BwKB\nDngVIcSPzxI3/lkIIQxDvDDf/5urOuZlhRDC7/d33IuZZMf3nyU5nU6n02l2FYUVDAbNLqHg\nLL+PNpv1mzp5vV6zSyg4yx9tbDZboT+M+l4XiS/6YGez20Q2u8NlY5quCZdrd7094mvn/POP\nj76zwXPM5Xddf27v1lwnZFk+8cQTW28OHDgwlUoVsGYhhBB2uz2TyXTAiJ0QYswQMaSPfcUG\nSQgxZ5G4/KR038qCv66qqoqipNPpjtlHU+T+6tK0jhgBNYUsyzabTdO0nT5j1iJJks1mS6fT\nZhdSQHa7XZKkDjismSj3Rt3791mnpiiKqqqZTMba+yhJkrWPNna7Xdf1TCZT6NdyOPa4THzR\nB7tAICBCLS1C9MjdjreEMu5upfZdHpbZNOd3d/75k8yIc2/+w4Xjeu68vzab7e67797xnoaG\nhkIWLYQQwWAwEol0WOg5/3jHnU/5hBC6Lh5+y/jp9EihX9Hr9SqKEovFLJx7XC6XYRjJZNLs\nQgrFZrOVlJSk0+lYLGZ2LYUiy7Lf749ECv6JMFEwGJRl2dr76Pf74/G4hTOBy+VSVTWRSFj4\njxCHw6GqqrWPNqWlpdlsttAfRkVR9hLsin7yRM+hw4LNy5fX529q3367Rhk5cujOD8qseOKu\nB7+uuHTWH//30l1TXRcxdliqX2X+kPfBYsfmRstexAAAAPak6IOdNHzK5O6rXn3kg5q0kd66\n8OFH50njpo52CSHiG7+eP//bRiFEetHrb2895sqbz+lv8Wuh9kKSxIxJ+emxui6em8dCFAAA\ndDlFfypWSINn3HzZ+rv+8KML/6BohmfItJ/+cLRbCCG2fvyPe57u9bPXbhm3YeW3yfTWP3zv\nnD/s8O9G/PDxX5/m28OTWtKEEakn5ro3NShCiLlfOy+anKgMWvYkKQAA+K7iD3ZCOAdOu+Pv\nJ25eXxN3VPTpHdx2prVyyvV3jXD0EUJUTvmfu0btermpp2dXG76TZTFjUvz+F3xCiKwmnvvQ\n9d9nR80uCgAAdJzOEOyEEEL1Vw3cpa+Gs2LQyAohhBC+quEjq0woqviccHjq6Q/cuQvs3vvS\neeHEeEXAshOsAADALor+GjscCFkW50/IX2mXyYoXP+ZKOwAAuhCCndWcOCpZGchfWvfW587G\nML9iAAC6Cr71rUZVdhy0k174iEE7AAC6CoKdBZ18dKq8JH9p3RsLGbQDAKCr4CvfgmyKMWNi\nPLedyUrPzutq84MBAOiiCHbWdMrRydYmdm9/4axrZiEKAACsj2BnTaoiLpq8fXrsMyxEAQBA\nF0Cws6ypo5K9yvODdrO/dNY0MWgHAIDFEewsS5bFzEn5K+2ymnjqA660AwDA4gh2Vjb58FSf\nivyg3dyvHRu3MmgHAICVEeysTJbFxSfkB+10XTz5PoN2AABYGcHO4iaMSPXvns1tf7jEsa62\ns6wODAAADhjBzuIkaYdBO0M88T7TYwEAsCyCnfWNHZYe3Cs/aPfJMseaGgbtAACwJoKd9UmS\nuGTboJ1hiMfncKUdAADWRLDrEkYfmh7aJz9ot2CFfcVGBu0AALAggl1X8b0psdbtx+cyaAcA\ngAUR7LqKIwdmRh6SyW0vWmVfst5mbj0AAKDdEey6kEunxFu3n2DQDgAAyyHYdSEjD8mMGpAf\ntFu81rZoFYN2AABYCsGua7ls6vYr7R59z2MYJtYCAADaGcGuaxnSOzt2WDq3vXqL+uESh7n1\nAACAdkSw63KuODmmbPu1P/KeO5M1tRoAANB+CHZdTq9ybcqoZG67rll5+wunufUAAID2QrDr\nii49Me6w5S+ve/J9dyIlmVsPAABoFwS7rqjcr591bH7QriUmv/yJy9x6AABAuyDYdVEXToz7\n3PlBu+c/dIWivBMAAOj0+DrvorwuY/rx+X7FibT0zDwG7QAA6PQIdl3XueOS5SV6bvvNha7a\nZsXcegAAQBsR7Louu2pcNDk/aJfJisdZZAwAgE6OYNelnXJUsk+Fltv+4GvHmhrV3HoAAEBb\nEOy6NFkWl52YH7TTDfHYewzaAQDQiRHsurpxw1PD+uRXn/h8pX3xWpu59QAAgINGsIP4/smx\n1u1H3vUYhom1AACAg0ewgxjZL3P0oHRu+9tN6ifL7ObWAwAADg7BDkIIccXJcXnbumKPvOfJ\naqZWAwAADgrBDkII0b9HdtLhqdz25gblzYX0KwYAoPMh2CHvipNiDlv+8ron5rrDcWnvjwcA\nAMWGYIe88hL93LGJ3HYkIT39b1qfAADQyRDssN2FkxKlvvwiY68vcG1uYJExAAA6E4IdtnPZ\njUum5PsVZzXxyHsec+sBAAAHhGCHnZxyVLJfZb5f8fyl9q/W0K8YAIBOg2CHnciy+PHp2/sV\n//1tj06/YgAAOgmCHXZ1xIDMMYPz/YrX1qjvf+0wtx4AALCf2j3Y6dEtS79csjFCh9vO7Ien\nxdRtEycee8+TTNP6BACATqDtwU5b/8Zt044Zf9eXQojE53dP7F014qjD+lQOm/nosmTb64Mp\nenfTTj0m/+trCMsvfky/YgAAOoG2BrvM57edcs5dH8T79A0IsfXJn9/2sTzhxj/+6daJ8eeu\nmnHPsnapEWa45IS4x5m/vO65D11bWzhrDwBAsWvrt/W8h//+bbfLn1345CX9Rfjt1/6dHXr9\nQ7P+65pfP//AdOeSV99Y2y5FwgQlHn3GxHzrk1RGenwu/YoBACh2bQx2kXXrGqTRkyd6hBD6\nRx98qPU+5ZRhQgjhHTnyEFFTU9MOJcIs54xN9CjNXyw550vnqs2qufUAAIC9a2Ow85SWOox0\nOiuEMBbOntvinTDhSCGEEMamTVuEx0OD287MporLT8oP2umG+Pvb/DoBAChqbQx28pgJ4+zv\n/+7GR155/KYbH97oOuXMKaoQRuSbe+56qtk3evTQ9qkSZpkwIjW0Tya3/c062/yldnPrAQAA\ne9HWa+x6XfnAb8dt/ceV5176wKfZw3/6i2k+0fzYqeWH//xD58l333Y6HdA6OUkSPzw1Jm3r\ndvK3tz3pLK1PAAAoUm2e6mg//MbZy5b+++Wnnp2z5ONfjbIJYTvk1Bvufuqz5W9fM5Q15C1g\naJ/sCYenctt1zcpzH9L6BACAItUel8MrpUMmnjMkf0OPhsvHX3LRoGFlDOxYxg9OjS1YYY8l\nJSHEc/NcU0elKoN0oAYAoOjQoBj7FvDqMyflZ1GksxKzKAAAKE5tHbHLfH7bKefcXXfozGk7\nNii+0PXGb3971YwhY7755bB2KbO9SVJHjCd2zKt0jGnHJ+d85VxfpwghPl5q/2KVffKRkhBC\nkiQr7eYucrtm4R3M4ZdoDexjp9b6RrXwPgqr72DrrhV6H/f+/JJhGG159jk/7jb11TPfXf3I\nSR4R/tdZpZetvn3pstuHiejzF1ZesOqXa768uX9bnr5QNK3gZxIVRemAV+lIn38rXfv7/BBv\nnwrx9O2Gwybput7Gt1Axy314rL2DsiwbhqHrutm1FJAsy9beQUVRRIcc1kyUe6Na/sNo+SOq\nJEmW/zB2wBHVMAxV3ePAXBtH7L7boPiKHRoUf1hTI0RRBrvm5uZCv0QwGAyFQlb6iA6sEMcP\n93+81C6E2FAvnpqtff80taWlxcJfJy6XyzCMZNKyFxXYbLaSkpJkMhmLxcyupVBkWfb7/aFQ\nyOxCCigYDMqy3AGHNRP5/f54PJ7NZs0upFBcLpfH44lGo+l02uxaCsXhcKiqau2jTWlpaSaT\nCYfDBX0hRVGCweAey2jbk9OguGv50elRpz0fVR99V91q5e9KAAA6HxoU4wB0K9Gnj0/ktuNJ\n8eDL5pYDAAB2QoNiHJgLJiRae528/Zn4Zh0LyAIAUCxoUIwDY1eNH52Wv0LCMMRDr7stfSEs\nAACdSbs1KB4cr19f/c1i3dWtz7jr75nYDk+LYjV2WProwekvVtqFEKu3KG997jxjjGWnFwAA\n0Im0vUGxMLbOu+fCIysDlQOGHX7EiMFVgcAhE6/6/Ye1lp0qCSGuOSNm3/ZHwT9ne8Lxdngj\nAQCANmrziF1iwc8mT71vuWfoSZddMuqQgNxSs/Lzd9565IYT3vviuYWPT+veHkWi6PQs06ZP\nzD41VxVCRBLSY7Pd150dNbsoAAC6urYGu00P3/J/y/r88M1PHjq1onXQxgh98euzTrj9ut9c\nfe4fxlq2xXRXd/lJ2dmL8h1P3vncOXVUamifjNlFAQDQpbXxDFrmw/c/zo65+vYdUp0QQgoc\nfevtF5Vunjt3RdueHkXM7RQ3TM9v64b4/SveLGffAQAwVRuDXSoe10R5efl3n9fv93TE+g4w\n08nHiDFD8qN06+uUFz52m1sPAABdXBuDnXfIkCqxYPbsyC73ayvnfLBR9OnTu21Pj6J37dnx\n1rUonpzr2tRAixsAAEzT1smMY3947aiWJ793wlV/eHNJXVwTQk/U/+f1+2ac9ssvXJMvnU6w\ns7ruQf2iyfHcdkaT/vCK10Kr4wIA0Mm0NdjJw25++akfVq14+PozDuvucThdDnflyLNufqF2\n8JVP/uvqXu1SI4rbeccnBvTIr8z9zTrb3K9ZbwQAAHO0vUGx3Hf6X788/qqXnnzh/UWraiKa\nq/yQIyadd9mM8VV8v3cNiiyuPyd6w18DuSUo/vaW9+jBmYCH9SgAAOho7bPQp737MTN+csyM\ndnkudEKDe2XPGJ14bYFLCBGOS/9423PT9F2vuwQAAIV2cMEu9u8HfvLMyn0+zDv5xvsvHHxQ\nr4DO5vsnxT9Z7mhokYUQc75ynHBE8siBtLUDAKBDHVywS/zntb/+dd4+H1amziDYdRUuh3Hd\n2dHb/+XP3fzja96/XBeyq8ykAACg4xxcsCu/+r3EVfu+hkpS7Qf19OicRh+aHjc8PX+pXQix\npVF56n3X5SfFzS4KAIAu5CCvsVPsTvqV4bv+68zo4rXBaEISQjz/kXvCyHT/bRNmAQBAobW1\n3Qmwo1KffumJsdy2povfv+LVmR0LAEBHIdihnZ0xJjm0T36U7ttN6hsLXebWAwBA10GwQzuT\nJXH9OVF126n6h99xb27kvD0AAB2BYIf2168ye/74/LSJVEaa9QInZAEA6AgEOxTEJVPig6ry\nJ2SXbbC9OJ8TsgAAFBzBDgWhyOKm8yK2bX3s/jXbvb6OE7IAABQWwQ6F0rdSu3hyIred0aT7\nnvdlNXMrAgDA4gh2KKDzJ8SH9M6fkF1Toz73odvcegAAsDaCHQpIkcVPzou0Liz25PvulZsP\nsic2AADYJ4IdCqt3N+2yqfkZspouZr3gy2Qlc0sCAMCqCHYouHPHJUYeksltV9crT77PDFkA\nAAqCYIeCkyVx03lRlz1/Qva5j9wrNnJCFgCA9kewQ0eoDGrfPzm/hqyui/tf8KU5IQsAQHsj\n2KGDnDkmedSgdG57U4Py2HvMkAUAoJ0R7NBBJElcf07U7cifkH3lE9dXq23mlgQAgMUQ7NBx\nKgL6j0/fdkLWEPc87wtFeQcCANBu+FpFhzrpqOTYYfkTsqGo/MBLXsMwtyIAAKyDYIeO9j/T\nIpVBPbe98Fv7K5/S/QQAgPZBsENH87mMn06PyNveeo+8415TQ/cTAADaAcEOJhjRLzNzUn45\niowm3fW0L5Gi+wkAAG1FsIM5Lj4hfti25Si2NCoPvekxtx4AACyAYAdzyJK4+fyIz52fOvHe\nIucHix3mlgQAQGdHsINpykv068+JtN7842veumbFxHoAAOjsCHYw0/HD06cdk8xtx5LSXc/4\nspq5FQEA0IkR7GCyq8+IHdI9m9teuUl9Yi5LjQEAcJAIdjCZTTV+dkHUpuYvtnvuQ/fXa1hq\nDACAg0Gwg/n6VWZ/cOr2pcbufs7XEOadCQDAAePrEwcjGo3OmTPniSeeWLp0abs84ZljkscN\n3b7U2F1Pc7EdAAAHjGCHAzZv3rzjjjvuyiuvvPfee6dMmXL55Zcnk8k2PqckiRvPi/Qozae5\n5Rtsf3uLznYAABwYgh0OTG1t7Q9+8IOrrrpq3bp133zzzaJFi6qrq3/5y1+2/Zl9LuP2iyMO\nW/5iu9cWuGZ/SWc7AAAOAMEOB+aVV14ZMGDA9ddfL8uyEKKqquree+994oknUqlU25/8kO7Z\n686Jtt588FUvy8gCALD/CHY4MHV1df3799/xngEDBqTT6ebm5nZ5/ilHpFo726Wz0m+e9sWS\nLCMLAMB+IdjhwPTu3XvJkiW6rrfes3jxYq/XW1ZW1l4vcc2Z0aF98svIbm5U7nveZxjt9dwA\nAFgZwQ4HZtq0aZFI5Oc//3kkEhFCfPnllzfddNPVV19ts7Vb8zlVEb+YESnx5DuG3fwAACAA\nSURBVLPjghX25z50tdeTAwBgYQQ7HJhAIPD4448vXLiwR48eJSUlp5566mmnnXbjjTe276t0\nK9FvuTAib3t7/nO2Z9EquhYDALAPXJmOAzZixIi5c+fW1ta2tLRUVVX5/f5CvMqoAZlLp8Qf\nm+0WQuiGuOc5/4P/FaoM0N0OAIA9YsQOB0OW5cGDB48fPz4YDBbuVS6cGD92SL5rcTgu3f2s\nL6MxkQIAgD3qHMHOiG/8Ys5rL7705kdLt6b3+sCN85+d8228o+pCYUmSuGl6pGdZa9di9Q+v\n0LUYAIA96gTBzmj4+L5r//ueZz765uu5j9x+zU2PLUns6aGJr17825OzCXYW4t25a/HsL53P\nf8RECgAAdq/4r7FLff74Xz71T5917yX97SK++M/X3vbnlyc/dFHfnR5k1C56ee7CpfPnfd4s\nhppUKAqkX2X2xmnRu5/LNz159F1P725a6ylaAADQquhH7LTFH34SGXnq2f3tQgjhPvyE40o3\nz59fvevDmqqXr2+Sq/pUKB1fIgpu4mGpmZPyA7G6Ie551reutvj/JgEAoKMVfbDbuGp1qtuA\nAb5tt/v37y82b9y4S8Naadi0W2+99dZbLz6cxUUt6ntT4hMPy69alkhLdzzhD8WK/t0LAEDH\nKvphj3A4LHbsp2H3+ex6Q0tUCN9e/tXO0un0+eef33rzvPPOu/jii9u1yt1QFCUQCBT6VUyU\nWyu2pKTE6Kh1Ie68Uvz4AWNZtSSEqGuW73om+OcbdHsh38KyLBuG4XJZ9qo+SZKEEE6n0263\nm11LoUiSJElSQadvm05RFCGEtfdRlmW/399hR5uOl/swer1ea++jJEkWPtrk2Gy2Qn8Y9/4m\nKfpgJ8SuuyBJQlEP9IxrbpmEnFQqJcsdMdjTMa9iltxhKPdB7ZhXdDnErGvEZXeL+mYhhFiy\nVrr7KeWOywt7EOzIHTSLJEmWf69aewdzrL2PrQccswsprK5wwLH2G1V0yAFnx1U9v6vog11J\noEREo1EhKnO3M9FoWi3xuw/kOex2+/vvv7/jPQ0NDe1X4u4Fg8FQKGThv728Xq/T6QyFQprW\ncU2DJSFuv0j9yd9KUhlJCPHmAtG9JH7+hD3Ok24jl8tlGEYymSzQ85vOZrOVlJQkEolYLGZ2\nLYWSG+kJhUJmF1JAwWBQluXGxkazCykgv98fj8ez2azZhRSKy+XyeDyRSCSdtuzMMIfDoaqq\ntY82paWl6XQ6HA4X9IUURdnLoGDRB+fuffs56teva30fVFdXG4cOHmxmSTDVwJ7Zn54fbf2b\n9tH3PJ8ut/jAPgAA+6nog53j6PGjnV+99caGrBDC2PrvFz5oGDJuTJkQQk/FwuGEZf98w54d\nPzx10eTtk2TvfY5JsgAACNEJgp2wH3fpj45qfPqn1/zstpt/fN3vFldd+uNTKoQQYuNLP7vk\nkt9/ZnZ9MMUlJ8THj9g+SfaXj/sbw8X/ZgYAoLA6wTiHVHnCrX8csPCTrzbG7KdcNfbYwYHc\nzInAiNNmzvT32emxlWOmzxx66AFdgIdOSZLETdOjtc3Kqs2qEKI+JP+/f/rv/0GLx2nZixoB\nANgnycJX9+8FkyfaLjd5orm5uSMnT+yiISzf8JdAQ0t+rO6IAZk7L22xtd9fK0yesAAmT1hD\nF5k8EQ6HmTzReTF5Amircr9+1+UtPlc+PX+9xnb/Cz7dslkaAIB9INihc+tbod1+Sdim5tPc\nvCWOh9/xmFsSAABmIdih0xvZL3PrjEhrP8gXP3a9PN+ya0UAALAXBDtYwbFD09ecEW29+be3\nPXO/YtlgAECXQ7CDRZwxJnn++PwSFIYhfveK76vVNnNLAgCggxHsYB1XnBw7cVS+uV0mK+58\nyr+WxsUAgK6EYAfrkCRx/TmRUQMyuZvxlHT7v/xbW3iTAwC6Cr7zYCk2Vdx2cbh/93yzq4YW\n+eePlIRivM8BAF0CX3iwGrfD+PXl4cpAvm3ypgbllof9kbhkblUAAHQAgh0sqNSn3/X9cMCj\n526ur1Nv+1dJIkW2AwBYHMEO1tSrXLv7yrDPnW9cvGKjeutj/mSabAcAsDKCHSyrX2X2zktb\nXI58tlu2wXbnU/6MRrYDAFgWwQ5WNqR39q7Lw057PtstWmW7+1mfpptbFAAAhUKwg8UN65O5\n7aKwTclnu/lL7f/3kk83zC0KAICCINjB+o4alLnlwoiy7c0+5yvHX97wmloRAAAFQbBDlzBu\nePr6c6PStuvrXlvgfGy2x9SKAABofwQ7dBUnHZm8+oxo681n/u16+F2yHQDAUgh26ELOOjZ5\n+dRY683nP3Q99p7bxHoAAGhfBDt0LTMmJS6aHG+9+cw896PvMW4HALAIgh26nEtPjF9ywvZs\n9+w819/fJtsBAKyAYIeu6JIpO2W7Fz92/e0tsh0AoNMj2KGLumRK/IqTt19v99J8sh0AoNMj\n2KHrumBCYpds91eyHQCgM1PNLgAw0wUTEpIQrX1PXp7vEob44WmxXMe7mpoaWZb9fr+ZJQIA\nsN8YsUNXd/6ExOUnbb/e7uVPXA+96X1v9pyjjz560KBBAwYMGDdu3Mcff2xihQAA7CeCHSBm\nTIzv2N/utU+dtz8ivn/FDzZs2LB27dpp06ZdfPHFK1euNLFCAAD2B8EOEEKIGZN2ut4u0H9G\njeenLk9pt27dfvKTn5xxxhm/+93vTCwPAID9QbAD8i6YkPivM6PytvVkv1hpv+EhV0tMEkKM\nHTt29erVZhYHAMB+INgB2515bPLWmWFhpHM3V2yQr/uTq6FF3rJlS2lpqbm1AQCwTwQ7YCfj\nhqen9HlNz0ZzN6vr5P/+k/vRp9+fOXOmuYUBALBPBDtgVzf9aOJA7X4t1Zi72Rxz9j9t3uBR\n55lbFQAA+0SwA3YlSdKf77/2Z+esdquR3D1Zw/2LR0sWrbKZWxgAAHtHsAN278TjBzz2c6V3\nNz13M5GW7njc/9J8l26YWxcAAHtEsAP2qDJoPHhtYnBVNnczo0l/e8tzy8Mldc18cAAAxYjv\nJ2BvAl7jnitbRg3MtN7zzTrbNX8Mzv7SYWJVAADsFsEO2AeXw/j1ZS1XnByzbVtaOZaUZr3o\nu+2f/qYInyAAQBHhawnYN0UWF0xIzPpBqHc3rfXOz1far3kw8Olyu4mFAQCwI4IdsL8G98r+\n6drQWccmpG2rU4Ri8q+e8P/uZW8iJe31nwIA0BEIdsABsKvGNWfG7rq8pbxEb73znS+c358V\nfGaeO5og3gEAzESwAw7YkQMzf/nv5smHp1rvCcXkx95zX3pf6aPveUJRPlYAAHPwDQQcDK/L\n+NkFkV/MiPjd2/vaxVPSs/Ncl94X/NPr3rqQYmJ5AICuSd33QwDswYSRqVED06984nptgSsS\nz5+HTWel1xc43/7cOfGw1AXj430rtb0/CQAA7YVgB7SJz2V8b0p8+vjEu184X/jY1dCSHwXP\namLuV465Xzm6B7UxQ9ITRqaH9snIXIMHACgkgh3QDlx245yxidPHJGcvcjz/kaumaft52Npm\n5dVPXa9+6irz62OGpMcOTR/eP2NTWZgMAND+CHZAu7Epxmmjk6ccnfzwP45n/u1aX7fT56sx\nLL+10PnWQqfLbhw5KH1E/8yAntn+3TWnnZAHAGgfBDugncmymHRYauLI1NdrbB8vdXy2wt4Q\n3mmWUiItzV/qmL/UIYSQJdGzTBvQIzugZ3ZAD21Aj2zAq+/hiQEA2AeCHVAQkiRGDcyMGpi5\n9iyxaov66TL7Zyvsa2t3/cTphtjUoGxqUOYtyS8+G/DqPUq1ioBeGdC7lWiVQb0yoFUEdJeD\ngT0AwD4Q7IDCkiQxuCo7uCp72dR4XUhZsNy+YLl9yXpbdg+TZUNRORSVl2/Y9X6f2yjzad1K\n9KBXLy/Rg169W4le6tPL/HrQq8t0LgIAEOyAjlQZ0M4+LnH2cYlMVlpXp6zZoq6tVVdvUdbX\nqon0PmbMRuJSJK6ur9vNj2RZdCvRuwe1yqDWs1TvXqpVBrQeZXrAw1ldAOhaCHaACWyqkRvG\ny93UDVHTqKypUdfUqJsblJomeWuLEo7vb3MUXRd1zXJdsyyEbcf7XXajR5k2uCp7aK/sob2z\nfSuyyg4DeytWrHjooYfWrl1bXl5+7rnnnnXWWe20cwAA0xDsAPPJkqgq16rKtQkjty9TlkhL\n9c1yfYtS1yxvbVHqQ/LWFrkpIje0yOnsfmW+RFpaW6OurVHf+UIIIRw2Y2DP7KG9sof2ykbr\nFvzw8jOmTZt2xRVXrFu37qabblq0aNGvfvWrAu0gAKBjEOyAIuWyG30rtd0uXBFNSI0RuSks\nN0bkxrDcGFFqm+TaZqWmSc7sOfOlMtLSatvS6tyo3tRjrth66DHOQ4+2T78gceqpp06dOvXc\nc8894ogjCrZDAICCI9gBnY/XZXhdWt+KXTOfYYimSD7h1TYrNU3Kuhqlul7VdnetXVL3v/GZ\neOMz4bC5DjtkzFGn/GbOh0sIdgDQqRHsAOuQJFHm18v8+vC+2+9MZaTVW9SVm9QVm9RvN6q1\nzcou/yqVEZ+vtIvKG+fUibUPZkcfmj56cGZon4zCTFsA6GwIdoDFOWzG8L6Z4X0zuZvhuLx8\ng3zTHY/3GHJ+ONN9lwevrVXX1qrPzBMuh3FE/8yogemjBmWqyvbQmgUAUGQIdkDX4nfrY4bo\nv76650UX9Z0+85pew2f8Z1OwNtZHUpw7PiyRkj5dbv90uV0IURnQjhqUOXJQ5vD+aZ+LPskA\nULwkw+iKh+lsNlvol1AURdOsPM4hy7Isy5qmWfgtJMuyEELXrdkNbvHixbNmzVq2bFllZeWF\nMy8dctSFH/9HnrdYrmve27/qWWaMHmIcNsA4erBREeyoWtvG8h9GRVEkSeqAw5qJFEXRdd3a\nRxvLH1ElSZIkyapH1BxVVQ3DKPQBxzAMm822p5920WDX2NhY6JcIBAItLS0W/s/r9XodDkco\nFLLwV6bL5TIMI5lMml1IoaiqWlJSkkwmY7FY653rapUvV9u+XG3/Zq26974qfSq0Ef0yhx2S\nHXlIptxfpAdrWZZ9Pl9LS4vZhRRQIBCQZbmpqcnsQgrI5/PF43FrH23cbnckEkmn02bXUih2\nu91ms+14tLEYWZaDwWA6nY5EIgV9IUVRAoHAnn7aRU/FdkzeMgzDwsEut2uW30dr72DOLvvY\nrzLbrzI7bVxC08WKjbbPVti/WmNbs0XVv/OfYUO9sqFeeWuhEEKU+vRBVdnhfTPD+2YHV2Vt\narH8R2t9o5pdSMGxj51aVziiCqvvYOuuFXof9/78XTTYAdgnRRatsy5CUfmrNbZv1tn+s962\nceuu82qFEE0R+bMV9s9W2IUQNsUYVKUd2jszrE92SO9Mt5IiHcwDAOsh2AHYt4BXn3x4avLh\nKSFEKCav2Kguq7btaSQvo0nLNqjLNqgvzxdCCK/LGFSVHd4nM7Aqe2ivbNBLzgOAQiHYATgw\nAY9+7JD0sUPSQohIXFpabftPtW1Ztbp6y+6vyYsmpK9W275anb/UtyKgD67KDuyZGVSVHVSl\n+d3kPABoNwQ7AAfP5zaOHZo+dmhaCKHpYtNWZekG29Jq2+rN6oatym6vA6kPyfUh+8dL7bmb\nuYvz+lZofSqyg3pm+1Ro0n4thAsA2A2CHYD2ocgit7jtacckhRChqLx8o/rtJnXVZtvKzWok\nvvu8tu3ivPzNEo8+sKc2oEd2QM/sgB7ZnmWaTM4DgP1GsANQEAGvftzQ9HFD870bGsPyqi3q\n6s3qqi3q8g228B5yXktMXrRKXrQqf97WpoqepdrAquygntlBVdmBPbMOm2Wn1AFA2xHsAHSE\nMr9e5s9fmacbYkujsnqLumaLunqLunqLGknsPudlsqK6XqmuV+Z+5RBCKLLo3S07oKc2sEd2\nQM9s/+5ZLythAMAOCHYAOposiV7lWq9ybdJhqdw9dSFl9WZl9RZ1TY26pkZtDMu7/YeaLtbX\nqevr1FzOE0J0D2oDe2oDemYH9swO7LnTlNtYLDZr1qwXXnhh69atgwYNuu6666ZPn17oXQMA\ncxHsAJivMqBVBrRxw/PnbaMJqbpeXbVZzZ293dig7GkVotpmpbZZ2WUqxqCe2QE9Mw/e/fPN\n6xffd999Q4YMmT179i233JJIJL73ve91zB4BgCkIdgCKjtdltPZGFkIk09LaWmVtjbqmRl2z\nRV1Xp2T2sNbZjn2SRa+neg7JfrrViHiVEccPuvPufrfe/MMLL7zQbrd32I4AQAcj2AEodk67\nMaxPdlif/CL3ub4qG7aq1XXKqi3qio1qS2z3p26bo+qC5WLBciGEX4gZh5w98ecPO0YNdg6s\nyg6uypb6aKEHwGoIdgA6mda+KuNH5O+pbVZWb1FXbc6fvd1TaxWbu8fSTWLppvzNMr/ev3t2\nQI/sgJ5a/x7ZHqW0VgHQ6RHsAHR63YNa96B2/PDtUzE+/br51//34tETLmlOdW+J7T6vNYbl\nxrD985X5M7Muu9Gve3ZgT61/92z/Htk+FZrLzpRbAJ0MwQ6A1VQGtHMm+SMb1Vtu6TdlypSj\nBxy1YElLTO897qSrNjWVhPZw3jaRlpZvsC3fYGu9Z8dVMfpWaP26azaFqAegqBHsAFjT9773\nvWOOOeaVV15patp83pRDLrpoWkmJJkRTXUjJnbRds0VZU6M2R3ef88R3VsWwqUbfCq015/Wt\n1CqDnL0FUFwIdgAsa8iQIb/4xS/8fn8oFGq9M9dapfW8bXNUXlujrqlR1mxR19aqmxv32Fol\nk5Vy7ZSFyHfRs6tGn21Rr3c3rW+F1r1UU/YYFAGg4Ah2ALq0oFc/alD6qEH5m8m0tL5OWVer\nrq9TquvVtTXqnlY/E0KkvxP1bKroWab1Ks/nvN7dtF7lmpNr9QB0FIIdAGzntBtDemeH9M62\n3tMUkbdFPXV9nbKhXkll9hj1MllRXadU1ymt90iSqAjoVWVar3KtV7dsbsmNbiW6xDlcAAVA\nsAOAvSn16aU+/ciB+W7JuiHqQ0p1nVJdr2yoV6vr9xH1DEPUNct1zfKXq7dPy3DYjFzCqyrX\nenfTqsq0qnLN42RgD0BbEewA4ADIUr67ypgh+XtyUW9DvbKhXtm4VdmwVdnUsMdeejmpjJRb\nFXfHOwNevXe5VlWez3lV5VrPMp15uAAOCMEOANqkNeqNPnT7naGYvKFe2bRV2digbKxXNjYo\nW0OKvteQForKoai8ZP32gT1ZFhUluZCn9yrLVpXrvcq1biWavG1+RiaT+ec//7lw4UJd10eN\nGnXllVc6nc5C7COAzoJgBwDtL+DRA4fohx2Sab0no0mbG5RNDcqmBmXjVmXTVmVzoxJN7G1g\nT9dFbbNS26wsWrX9Tpti9CzTq8q17sH0y0//taVuybTTj/M6Uk8++eQzzzzz7rvvut3uwu0X\ngCJHsAOAjmBTjH6V2X6V2R3vDMXkXMLb1KBsaVQ2bVW2NMoZbW9pL6NJ1fVKdb0ihF30udXf\nR8ypFx6nOOzCn61Z9u///u3yC88d17NUqyrXgl4WwwW6HIIdAJgm4NEDHn1Ev+0De7ou6luU\nzQ3K5kZlW+aT93kaN5YUa2psIjh1sy4eeDF/p8tu9CjTepZqPcv07qVaj6DWo0zvVkKnPcDK\nCHYAUERkOX/FXmtrPSFERpO2NMi5Ub3NjUrulO5e1szISaSltTXq2p2naCiyKC/Repbq3Uu1\nyoDWvVSvCGjdg3qpj+E9wAoIdgBQ7GyK0bdS61up7XjnL//33oWLa2++7cGmuG/jVnndluzK\nDQlD9u39qTRd1DUrdc2KWGPb8X6banQP6pUBrSKoVwa07kG9IqBXBLSgT2fZNKATIdgBQKf0\nkxt+PHny5D/86oyrr776MJ/6xfxHqr/88s13P06Jii2NypYmZUtj7n9yOL7vk6+ZrLRxq7Jx\nq7LL/TZVlPm0ioBeEdArg1pFQC/3a+UlemVAZ0UNoAgR7ACgU/L7/e+99979999/1113aZo2\nevTouXPn9uhRKsROK2cIIeIpqbZZqWmUc3NsaxrlmmalvnkfszRyMtn8zNzv/sjnNsr9+cxX\n5tO6BfSKgF7u18tLaL8HmIZgBwCdVVlZ2W9/+9tgMCjLcmNj454e5nYY/btn+3ff6U7dEI1h\nua5ZqWuWa5uVupBcH1Jqm+SGsJLV9vBEO4vEpUhcXVe7mx8FvPmEl8t85X69W0Av8+ndArpd\nPZjMV1dX53A4ZJl5H8A+EOwAoCuSJdGtRO9Woo/ot9P9ui4aI3J9SKlrlutD8tYWpT4k17co\n9c1yIr2/V9vlmi2v3rKbH/ncRu7cbplfL/PrFSV6mT9/ene3i6o9+uij9957b0NDg6qqJ510\n0m9+85uqqqoD3Vmg6yDYAQC2k+V84Bved9cfReLS1rBSH5LrQ3JDi9IQluua5Yaw0hiW93OQ\nT2wb51tft5sfOe1GRUAv8+llfr28RCv360u+nPvUY2/+4vb7zz93SnV19Z133jlz5szZs2c7\nHI427SRgXQQ7AMB+8bkNn3vXU7pCCN0QzRG5PiQ3RZStLXJ9SG4My1tb5K0tSlNE1va7j0oy\nLeWW3N3hvrOrTjz7yf+I51eIMn/PygmvbV7w9i/+XDd+zKG5Ab8yn1bq0218lQHb8GkAALSJ\nLIlczBIiu8uPdsl8DWG5MSzXh+SGsNwU3q/ZGznprKhpkmua7I4eZy9tEEvf3OmnAa9e5tPL\nS/Qyn17q00v9+Y0yvx7w6FyYhy6FYAcAKJS9ZD4hRCgqN0bkrS3y1pDcGJEbWpTGsNwYketD\ncnK/r+cT2y7pW1OzuwJkEfTqZT49mMt8Pj3g0ctL9BKP3q1ED3gNJvDCYgh2AABzBLx6wKsP\n6LGbH8WSUkOLfOv/PhhNOs++4EeJrK+2UVu9Ibq1RSj24P6/hK6LxrDcGN7jqJ3fbQS9esCr\nl/n1Eo9e6tODXj3oNXI3Szz0Z0YnQ7ADABQdj9PwOLU/3DnjvPPOu//GWWPHjt2yZcs333xz\n3333TZs+s6FFborI9SG5OSpvbZFz0a0xLDdFDuD0bk44LoXjSnX9bhr1CSFkSeTiXZnfCHj1\nEnc+/wU8Rqkvn/zU3f/TA5DJZBYtWtTQ0FBVVTV8+HBJIkvi4BHsAABFqrS0dM6cOe+8887q\n1avHjRv3l7/8pXfv3kIYPcu0nmW7n4gbjstNEakxrDRFpMaI0hSWGiNKU0RqCstNUTmTPbDM\npBuiOSo3R+XdTuPN8bvzmS/g1YM+w+/WAx496NUDXqPEowe9u2/j0mr58uVXXXVVbW1t3759\nV69effjhhz/88MMVFRUHVCfQimAHACheiqKcfvrpfr8/Ho9ns7u5UG8Xfrfud4t+lbuPfZGE\n1BSRQ1G5ISy3xOSGFrklJjdF5KaoHIpKLbGDmWeRG/MTYo8DdzbFKPEaJW496NVzGwGvHvQa\nfrfusiV/fM1tY46d+LsHfltWVlZdXX3FFVdce+21zz333EFUAgiCHQCg6/C5DJ9L61ux+9iX\n1UQoJjdF5OaI3BKXm8Jyc1TK3dMSk5ujciR+MCdJM5rU0CI1tOw2Nfr9Y/+9XIiz7xABryjx\n9Op34qsfzH7p/meyfat8AY/hc+t+t17iMYJe3e1gngf2jWAHAIAQQqiKKPfr5f49dt7LZEUu\n4YVicigqhaJyc1RuicuhaH4gMBw/gL59O0qmRW2TqG1ShPCWDrp0zhIhluz6GJsqfK58zgt4\ndL/H8Lt1v9vwufQSj1Hi0XM3nXbyX5dGsAMAYL/YVFFeopeX7C27tcTkUEwKx+SmqNySy38x\nORSVQ1EpHJdDMTmaOMi5EZmsaIrITZF9nC+2KYbfY/hcut9t+N258Kf73IbPnb/Tm/t/J42d\nrYnfKgAA7abEo5d4hBB7XGQtkxUtcTkcl5siclNYn/W7h72BXmPGn54V3s21sTXVjZKjmyG5\nDrqAjCY1hqW9dHhp5bIbvm05z+PMZT7D59K9LiN3j89l+FyGx6l7XW0dBVyzZs2CBQvS6fSI\nESOOOeaYNj4b9oJgBwBAx7Gp+RO+ucXZhv7mhOuuu+7+a2eWlpY2NTWdc845s+6Y5XT7W2JS\nOJ6f0hGOy+G4FI7LzRGpJS5H4nJLTIokDmCJ3t1KpKVEWqoP7TsCSpLwOg2PU/e5Da/L8Dp1\nr8vwugyPI7+x47bHadjVnYLg73//+3vvvfeII47wer133HHH1KlT//rXvypKm/vEYHcIdgAA\nmKZ3794vv/xybW1tY2NjZWVleXm5EEIIo9xv7OVqv5x4SgpF85mv9f9bY19uOxyX0gfY5OW7\nDENEElIkodQ279fjbYrhcRoel+FxGplE0+IvR13+y1UDD+nuc0unf3/rX/886//dP+fC6Wd4\nnIbPbbjshtvJEiDtRjKMrvifsqGhodAvEQwGQ6GQhf/zer1ep9PZ3NysaW37m7GIuVwuwzCS\nyaTZhRSKzWYrKSlJJBKxWMzsWgpFlmW/3x8KhcwupICCwaAsy42NjWYXUkD73+6kk3K5XB6P\nJxwOp9Ppdn/yVEYKx6VIQo7EpWhCbolL4ZgUTcrhuBRNyJGEFIlL0aQciUupjGm9kW2K4XYa\nubPDbofhdhhup+F2GC674XXpTnt+2+0wvC7D7TAcdsNlN/beI7CDybJcWlqaTqfD4XBBX0hR\nlGBwj+uvMGIHAICVOWxGtxKj217nfORkNCmakMJxKZaUw3EplpCiSTmSkKIJKZqQowkpkpCi\nSTmWkKJJ6YDW892fl26JSS0xsZ+Dgq3cDsNpzw/7eZyG02Y4bIbHaTjt+f95nYbDbjhshsdh\nOO2Gwy5yGdFuM1ztN4N46dKl99xzz9KlS0tKSk488cQbbrjB7Xa315MfEIIdAAAQQgibYgS9\nRtAr9jL5o1VWE60hL5oLfEkpmpBi2zekaFJav7EhlbE53GWJdg2CreIpp5FVfQAAIABJREFU\nKZ46+Gd2Owy7ajjthsth2FXhdhoOm2FTDJ/LsKmGwybcTkOR8xcOOmzC5TBU2fA4DZsqnHbD\nrho21Vj97eIzzzht5syZV199dUNDwwMPPPDJJ5+88sorqmpCyiLYAQCAA6YqIuDRA559PGzz\n5vrJkyfPmDHj+p/8j+oIPP3cG7P+7y+33XHP4KFHxlJSLCnFU3I8KUUTUi6ixVNSPCnFUlI8\nKcdSkn5QfQH3Xz4XtvVqlBNGXJZc5zKeWyI9fFN60qRJEydOfPbZZy+++OJ2KfKAEOwAAECh\nVFVVPfPMMz/96U8feughIUSPHj3u/dWvzj13hBD7dTVhKiMl0lIilU9+iZQUT0nJjBRNSImU\nlExv306kpWRaiiXzd7bvmeL9EU1ImawQQvh8vpNPPnnhwoUEOwAAYDVHHnnk3Llzo9FoMpnc\nNu13fzlshsNm7HNccLeiCSmZkVIZKZ6UEmkpnZESaSmeklJpkcxI8ZScSotURoompUxWSmak\neFLKZEUuOGayUix5wNHQac9vpFIpu92+18cWCsEOAAAUXFlZmaqqHTkHP9dXry3PkMlKyYyI\nJyVNl2JJKaNJqbSIpyRNl6JJSdNEIi2ls9KLL72h2L0TJpzQu7tTCLFx48bXX3/9wQcfbKf9\nODAEOwAAgN2wqYZNFb59pcPJhx46derU+Ip+Z5999s/mNvzzn/885ZRTTjnllI4pchcEOwAA\ngINXVVX1ySef/OMf//jss8/8fv+sWbPOPPNMs4oh2AEAALRJIBC4+eabO6ZB8d7te4U4AAAA\ndAqdJdhpkdq1K1dtaEru5Tz3/jwGAADAsjrDqdhM9Zv33vnIZ43CZmRtvU/679uuGVex6xTk\n/XkMAACApRX/iJ2x9rl7//Ztvx/+6ekXX3jqgem+ef/34Oymg3gMAACAxRV/sFv27jsb+591\nxcm9nUJyD5h23lj1mw/mNx74YwAAACyu6INdzfIVLSVDh/bM35SHHDrI+HblygN+DAAAgNUV\n/TV2TU1NoqysrPW2p8SvZFvCcSHc+/2YbDZ77733tv74uOOOO+644wpduCzLHs9BrYHSSdhs\nNiGE2+02DMvOVlFV1TAMVS36j8nBkmVZCGGz2bxer9m1FIokSbIsW3gHxbbfo7X3UVVVt9ut\nF3pBePPkjjMul8usdag6gKIokiRZ+I0qSZIQQlVVc/ex6L+x0pm0UBRl+x12u01oWe1AHqNp\n2ksvvdT6U7/fP3ny5ALWvI3T6eyAVzGXw+Ewu4SCy0VYC1NV1cLhNacrfBgtv48WTjytbDZb\nVzjgmF1CYcmyXOgP497/win6/74l/hKRSCRab+uJREry+70H8hi73f7qq6+2/tjn8zU3Nxe0\naiGE3++PRCIWHs1yu90OhyMcDmuatu9Hd0652JpKpcwupFBUVfX5fMlkcsfPj8VIkuTz+czt\nF1pofr9fluVQKGR2IQXk9XoTiYSFjzZOp9PlckWj0UwmY3YthWK32xVFsfbRJhAIZDKZaDRa\n0BeSZbmkpGRPPy36YNetslKq27QpK/rkSq3bUmP0GtNHOpDHSJJUVVW1479oaGgofOlC0zQL\nB7vcrmmaZuFDrWEYhmFYeAdzp/Asv4/W3sFW1t5HwzB0XbfwPubGYCy/j7IsW3gHi+SIWvST\nJ3xjjh2hffXRgrgQQojMug/mr68YM7rvAT8GAADA6op+xE6UnHDhaa/e/sdfPLD+uG5NX8ye\n23L8T88eKIQQNW/95q63Kq/645VH7PkxAAAAXYfUGc4VGqFlb7/w1qJNcXuPEVPPPevIClUI\nIerm/v73c7td9JuLRuz5MQAAAF1Hpwh2AAAA2Leiv8YOAAAA+4dgBwAAYBEEOwAAAIsg2AEA\nAFgEwQ4AAMAiCHYAAAAWQbADAACwCIIdAACARXTR9RkaGhoK/RLBYDAUClm4/7PX63U6nc3N\nzRZe0dn1/9m77/Aoqr0P4GfK9p7eA4GELiVKb0q/CiggYEWv9Vqv/VUv2LvCBVHsKJZrAbEi\niIiFJk2kNyEhhSSkbK9T3j92E4iSkLC7md3J9/Pw8MzsTmZ/k+xOvjlz5hyNRhRFr9crdSHR\nolAoTCaTx+NxuVxS1xItNE0bjUar1Sp1IVFksVhomq6pqZG6kCgyGo1ut5vjOKkLiRaNRqPT\n6ex2u9/vl7qWaFGpVCzLyvtsk5CQ4Pf77XZ7VF+IYRiLxdJkGVF9bQAAAABoMwh2AAAAADKB\nYAcAAAAgEwh2AAAAADIRgzdP+LcveWJLp3tvGmI63bOi7eD3y7/dWuRgU7oMm3Tx4CxV848D\nAAAAtBcx12IXKP3us6//OFITOO2z/v3vP/R/b28LZPfsllK7du69j399XGzucQAAAID2I4Za\n7MT9y594d/WBA6UOnnQ7/SZ1P7z/RU3hHW/eP9JACBlmuvvGj5fv/Mctve1NPM60Zf0AAAAA\n0oqlFjtDdp/B42dcO6GLsokNvDu27iF9Rww2BFfTzjs327F58/4mHwcAAABoT2KoxY7KPHdS\nJiHkj+KPvis97RYlxcVC6qjshtyXkZFJ6qpO8E09TkiwyU4URYfD0bAblUpFUVS0DuMUbfMq\nUgkeHUVRMj7MhmOUupDowg9RHmR/jO3kjSrvY5T9Af5lIdovdFoxFOzOzOVyEa1W27DO6LRq\nsdLhbOpxQoL3X/j9/gsuuKDh6Wuuuea2225rg3oTEhLa4FWkZTabpS4h6nQ6ndQlRJdarVar\n1VJXEV2JiYlSlxB1sj9GpbKpqznyYTAYpC4h6mR/tlEqldH+MAqC0MyzcRXsFEoFaTSjDC/w\nRKPRNPV4/SpFUf379294Njs7OxA4/c0ZEcSyrIxnvyGEMAxD0zTHcTKeNo2maXKmj1BcoyiK\nZVlBEGQ8LxxpBx9GlmUpimqD05qEGIYRBEHeZxuGYXiel/EJh6ZpiqLkfbZRKBRtcEYVRbGZ\nv3PiKtiZzWZitdkISQ+uu23WgDY5QWmuO/3j9V+nVCpfffXVU/fUNnPF2u12GZ+GgnPFOhwO\nGX9K28lcsT6fT96zNxqNRpvNJnUhURScK1bex9hO5op1uVyYKzZ+BeeK5TiuDeaKbSbYxdLN\nE2eU0a27pW7fvqrQKn/gwJ9Mr17dmnwcAAAAIPpqamoee+yxsWPHXn755Z9++qmEzTpxEOzc\nJTvWrz9QQwiheow6P+3Ql++sPe4X/Sc2v734Z2rImP6aJh8HAAAAiLKSkpLBgwdv37593Lhx\n+fn5jz766E033SRVMXFwKfbEuree+1/WA1/93xBCFcy8f1bRUwtumrGA4UVd1yn33dhfS0iT\njwMAAAC0gsNDcTzlC1BuL8XxxOWlOIHy+imvnwT40INuHy2IxO2lOIF4fNSm39zdL/4pP7+b\nJ4V97Hb/5ZdfPnLkyG+++eaiiy5q+/qp2OsE5izbe9SX0jUvSRFc91YdOlSpyumVUz/FGGcv\nKzruVqXkZFsazRvW1OOn0TZ97KxWa+x9eyMm2Meurq4OfeziV7CPncfjkXevF6PRaLVapS4k\nioJ97GpqaqQuJIraSR87u90u1z52W7ZsWb9+vc/nO+ecc8aPHx+lAUEEkbi8wRBGef2U00v5\nApTPT7l8lMdH+TnK46PcPsofIB5/cIHyBii3lwrwlMdHeQNUILy3WPcO5K17/Ha7/Z577qEo\n6sUXX4zQkTXCMIzFYmnq2RhssdNndu916ro6Jb9XyqkPsMbMzsbTfGFTjwMAAIBk5syZ8847\n74wePVqn0y1evPiNN9745JNPmh+/xuOjXN5QIHP7KJeXdvsot49yeymPn3J7KZeX8vgpj5/y\n+imHm/L6KY+f8gUkHiTPVx/LlUqlx+ORpIYYDHYAAAAgEytXrvzwo0+Xf/Nzh7ye3gA788aa\nOY+9cPcz60ZccKHLQ7l9tMtLhf75KJeXcntpl5cS4uFyl4IR1UqiYMXa6uNqFZWdmdIxgyWE\n2Gy2FStWPPjgg5JUhWAHAAAArePnKIebcngop4d2eihH6H/K6aVdHsrppRxuyuWjnR6q1nZp\n/owrZ3/W8KUZqm7zDvvJ4ZXSVE7TRKcSVUpRxYpatahWikpW1KpEtZIoGFGnERWMqFaKGqWo\nVBCtSlQwokohqpSigiE6tcgwRKsSFAxRK09mzz17Do8fPz7nkksKe0x9663KBQsW5OXlTZ8+\nXZIDRLADAAAAQgjxc5TdRdk9tN1FOTy0w03Z3ZTTS9vdlNNDO9yUw0M7PJTdTQW4Fl/0pM7U\n7b31VApRoxTVStGgDYUwjVLUqUWtWlQrRJVS1KlDG6gVok4jqhSiWkm0KkGtEBVRCD49evRY\ns2bN3LlzH3roIaPROG3atJtvvjk4xH3bQ7ADAACQOa+fsrkom5u2u2m7i7K7abubcnhoq5Oy\nu2mHOxTm/C2PaxHCMkSrErQq0aAVdWpRpxa1KjH4iFZ96iOiRiVqVaJOLehUokSRqTkFBQVv\nvPFGQkKC3++P9gDFzUOwAwAAiGMOD2Vz0TYXbXVSNjdtc9E2F2V30zYXVeekgwutaGALD00T\nvVrQa0S9WtRrRJ1aUNLeLz7/oENWwqiR/RNMyi0bf/zs48X/nfvkgMLuOrV46gVNiAgEOwAA\nAMkEAoG33377o48+Ki8vz8vLu/nmmy+55JKG0UACHLG56ToHbXXRVidlddK1Ttruoq0uus4R\naoELc4SOltCoRKNW1KsFo1bUawSDVjRoRL1GMGhOLujUglEralSnCWoX9+v3n//8Z/bN13Ac\n17179zfmPTJyZFdCZDsrrrQQ7AAAAKTB8eS+h19Y/9v+y2a9mJiWv3Nv+Qsf7/9uf7U5uXMw\nzzk8UWxpU7DEoBGMWtGgFUzB/3WiQSMYtKJRIxh1J5Mcy4T1Qvn5+Z988kmwz5kgIM9FF4Id\nAABAtDjcVK2DrnPStQ7a5qKr7bTVRdc66DoHbXVSVhdNFHMTh5LvjxByhBCSb8ofUeIiJWEP\nGW7UCiadGPzfpBNMukbRzawXjVpBe7rWtehRKBQsy8p4OPQYgWAHAABwlgSRWJ10nZOusYf+\ntzrpajttddI1DrrWTgX4yDe5GbWhrGbWiRaDYNQKJq1g1osWvdCQ52Lw9gJoGwh2AAAATRJF\nYnXRdQ76hI2uddA1DrrWQdfaQ8tWJ81H9NIiTRG/uzI7XZeZrDTpRbNOSDCEMlyCUTBpBZMu\n3AujIG8IdgAA0N55/NQJK13roGvsoQBXbWesTrrKRludNBfRCbENGjGY1RKNgpJyvv/OvMun\njxt7wXnZqWoF5Xhz0fNL3nt35ZYtLItf0HA28L4BAIB2we6mauxMlZWusdPVdvqEjam207V2\nuspGe3wRu2DKMsSsExKMQoJeSDAICQbBYhAS65OcxSAqmEY929L5tHvuGX3ooot69Oixfv36\nTZs2ffLJJ0h1cNbw1gEAAPnw+KgqK33Cxpyw0dV2usrKnLDRJ2z0CSsdqdF3FYyYYBQTDHyC\nQUw0Cgl6IdEoWPRCkkkw6wSzvnWXZmfOnNm7d++lS5fu37+/X79+8+bNy8jIiEid0D4h2AEA\nQOzat2/fvHnz9u3bl5CQMHHixGuuuYZlWUEktQ66yspUWelgjKusC2U4lzcC6Y2miUUvJBqF\nBIOQbAqFtgS9kGTiEwyiURvhATu6dev29NNP63Q6u93u9/sju3NobxDsAAAgRm36betls/49\n5sKrJ193b/kJ4YN1lV8dqLakdquxM+H3e1MpxBSzkGgUEo1CUuh/PsEQynC4qxTiFIIdAABI\nLMCRUKubjamorf/fylRZx3SZuv8YIccOEUKIOpP4Camsa8We1Uox1SwkmYQEA59iFpKMQpJJ\nSDbxiUbBoMFkViBDCHYAANBGOJ6csDEVdXSVNRTdjtfSVVam1kELp0tZVMsuq7IMSTTwKWYh\nxSIkm4QkI59sElItQqKRR3qD9gbBDgAAIkwQSa2drqhjKuroijqmopaprKMrrUy1nQ5nQimV\nQhQ8pXqlY/C5eckmPsUspFqEVDOfYMCVU4AQBDsAADh7Tg9VUcccr2Uqak8muao6OpwZFzQq\nMdXMp1mEjT9/npOmmHXZ2I6ZapPafeL4wfPPP/+DDz4YOjQlgocAICcIdgAAcGaCQE7YmfIa\nuqKWqahjjtfSFbXM8TrG4T77AKdViakWPtUspFr4NIuQbObTLEKqmTdoQ9dP9xZmTJgwwVM0\nbMKECSUlJYsXL542bdrQoUMjdEwAMoRgBwAAjQR4qqKWPl7LlNcw5TX08VqmvJapsjIB7ix3\nqFKIaQlCmoUPXjlNswipFv7UANeU7t27r1+//pVXXlm2bJnFYnn22WcvvvjisywCoH1AsAMA\niG+iKPL8WQ7+wfGk0sqU1zCl1Ux5DVNWzZRV0ydszGlvZTgjliHJJj6U4cx8eoKQauHTEgSz\n7uw71mVlZT3zzDNGo9HtdnPc2UZLgHYDwQ4AIF4dPHjwkUce2bhxoyAI55133iOPPHLOOec0\ns32dky45wZRWM2XVTMkJpqyaqbSe5YBwRq2QniCkJfDpFj49UUiz8GkWPtmEmxgAJIZgBwAQ\nlyoqKiZOnDh27Njvv/9eoVC88cYbkydPXrNmTV5eHiGE40lZDVN6gi05wZScYEqqmbJq5ixm\nZaApkmwW0hP44L/MRCE9gU9L4LUqDCMCEIsQ7AAA4tLcuXP79ev38ssvWywWX4D+52355d7e\nD8wv7j2gT3EVc7y21U1xNE2STXxWkpCZyGck8hmJfHoCn5Yg/GXSegCIZQh2AABxxhegjlUx\nu8pzuhTePGeJsbSGOV5DRNFMtLcRQtbtadFOkkxCVhKfkcBnJPKZSXxmIp+eiAwHEPcQ7AAA\nYlqAp0qqmKJKpriKLa5kiiqZKisjiITkPrq/jpAWzK+lVopZSXxmEp+dxGcl88FljRIZDkCG\nEOwAAGIIL5DyGqaoMpThiirZ47UM35qbSk06ISeFz07mc5L5nBQuO5lPNoUx2wMAxBUEOwAA\nKVXb6eJK9kgFU1zJFlUyxVVMgGvFLQ6cp8KkqtFSFft+/zYnmX9r4UMJRiZ61QJAjEOwAwBo\nOx4/VVzJHKlgiyrYo5VMUSXbqpkbzHqhYyqfncx1SOVzUvjcFG7T+rVbt27leX7GrX0uvPBC\nijr7eSAAQAYQ7AAAokUUSUUdc+Q4c7SSPVrBHK1gK2pbMfavXiPmpnC5qXyHFC43le+Yxhu1\nf72oOmbMmOnTp9M0XVNTE+HqASAOIdgBAESMn6OKKpg/j7NHKtg/y5mjlazH19ImNCUr5qby\nHVK53BS+Yxqfm8IloW8cALQSgh0AwNmzu6k/y9nDx9k/y9mjFWxpdUtvdKApkp7Ad0jjOqby\nHdL4jqlcegKPaRsAIEwIdgAArVBjp49VMcVV7KFy9nAZe+wEI7bs0qpOLeamhPrG5Wdyeekc\nBhwBgIhDsAMAaE5lHR3McIfK2cPlrM3Vola1YINcXjrXKYPvmMp1TONSzLiuCgBRh2AHANDI\nqUnuUBlrd7coyakUYodUvlM6l5fOdUrnO6ShQQ4AJNBOg13bjAgg73EHgkdHUZSMD7PhGKUu\nJLrwQ7S56INl7P4S5mApe7CUtbasTc6gETtlcJ0zuM4ZfKd0Livp7z3k2vS7KuMfYgMZH2N7\nOKMSuR9gw6FF+xib3z8ltrB7iLxwHBftl2AYhudbOQV3XKFpmqZpnudl/BaiaZoQIgiyvYJG\nURTDMIIgyPgYyek+jB4f2X+M2ltM7S2m9haR47UtOgsnmUjXHLEgSyzIEguyxYzE6JTbegzD\nUBTVBqc1CQXfqPI+28j+jBpMdfI+27AsK4pitH/7i6KoUCiarCGqrx2zrFZrtF/CYrHYbDYZ\nf0T1er1arbbb7TLOrxqNRhRFr9crdSHRolAoTCaTz+dzuVxS1xIV27ZtW7ZsmdVq7dChw4VT\nbjzuSD1Uxh4qYw+VKwItSEEGjZidzOVncvmZXH4Gl5va6K0e/bNIS1ksFpqm2+C0JiGj0eh2\nu2UcXjUajU6nc7lcfr9f6lqiRaVSsSwr17MNIYSm6YSEhEAgYLfbo/pCDMNYLJamnm2nwQ4A\nZG/+K0sWLfml79Cr6JRzfqxN/3Gx/oxfYtSKBZmBgiyuIIvLz+ASjXJuWgAAWUKwAwCZ4Hhy\ntILdU6w4VM7uPSoct97VYfTddYSQACFNzJ6qVop5aSfb5HJSePn2/wGAdgHBDgDiVTDJBe9d\nPVDCFlexZxwcmKZITgrfJSvQLYfrksXlpnAYExgA5ATBDgDiBi+Q0hNMMMkdKmMPl7N+7swt\nbHoNyc8MdM30PvnwlZ+983CfXnltUCoAgCQQ7AAgdnE8KaoMZbhD5ezRCibQgiSnVoop+tod\n6z986M7Jg/uYCnL1Vqvt008/pe3renTNboOyAQCkgmAHADEkwJGjlezh+iRXVMm25PZVhiY5\nKVyXLK5rNleQxeWmcAxNbti/9pn75993331VXbuuXr36v//979y5c5sZIwAAQAYQ7ABASn+5\nunqonG1JmxxNk+wkvnMml5/B5WdynTM4leKvQwstXLhw0aJF8+bNq6qqys/Pf/PNN8eOHRud\ngwAAiBUIdgDQpgSBFFexB8vYg6XsoXL26HEmwLcoyWUl8fn1Sa5TOqc+04RdKpXq3//+9913\n3200GuU9xhsAQAMEOwCIuhr7ydlX9xQrnJ5WtMnlpnA5KXyP3IBBI9vhvgEAIgXBDgAiz+On\nDpSwe48pDpSyB0paNPsqTZOcZD4/M5CfweVn8nnpp7m6CgAAzUOwA4DICDbL7S1W7C5q0Zxd\nf+kn15KrqwAA0DwEOwA4S6JIiquYnUcUu4oU+44pqu1nbpZLNfNdsrkuWVyXLK5zBpIcAECE\nIdgBQCs0hLk/jip2FylsZ7rGatCKXTIDXbK5gkyuSzZn1mH2VQCAKEKwA4AzO17LbD2o2HHk\nzGGOokhWEt8tO9CjA9c1O5CTjNlXAQDaDoIdAJye10/9cUSx7ZBy6yFFeQ3TzJZKVuySxfXo\nwHXLDnTLCRi1uMAKACANBDsAaORoBbv1kGL7IeXuIraZEeaUrNgthzunY+CcvEDXbE7BIMwB\nAEgPwQ6gnfL7/V9++WVxcXFSUtKw4RdUebI37Vdt2qc8YWvySmswzPXqGOiNMAcAEJMQ7ADa\no/Ly8qlTp/p5dbeB11fv7/7ujlxaYWhq49xUvrCz/9wCf68OnIJFmAMAiF0IdgDtC8eTIxXs\nvY/+mjxkGa85p1oghJC/t9HpNWKfPP+5BYHCfH+yCbeyAgDEBwQ7APmzuuj9x9i9xxT7jrEH\ny1hfgCIptxJCyN8CW4dUfmBXX/+ugS5ZAebMw9IBAEBsQbADkBteIJV1THkNU1LNHC5n9x1j\nm7+nlaFJzw6Bw9veKezkfOiOWW1WJwAARByCHUAcC3CkxsFU1tFl1UxZDVNWzZRWMxV1DMef\n+WuVTGBwTzKqUFGY73XZjg94afZ1F70Y/ZIBACCKEOwA4oDHT1XUMpVWuqqOPmFjqqx0lY2u\nsjJ1Dlpozc0MSSahe06gWw63e9NHnyx+7OaBTxd2HH9gz4GHHnooLy9v7NixUTsCAABoCwh2\nALHF7qbLqunjtUzwX3kNXV7LWJ1n2d9NwZLOGVzXrEC3HK57TiCp/jaIyQMnJ6qP33XXXbNm\nzVIqlZMnT3788cdZFicEAID4hvM4gMTqnPTBUvZQGXuwjD1Yxp51hiOEJBmFzCQ+M5HPSOSz\nkvisZD7NwrOn619H0/Qtt9xy5513ejwerVbr9/vP/gAAACBmINgBtDU/R+06qmhIctVNDwjc\nFKNWSDELKWYh2cSnWoQUs5CewGcm8mplqweZS0tL83g8CHYAAPKAYAfQRpweavMB5fq9yq0H\nlb5Ak1N1nUqtFNMT+IxEIT2BT0/g0yxCsolPSxCUGCUYAABOB8EOILrqnPTGvcr1e1V/HFE0\nf7OqUSsUZHKdM7jMJD4jUchI5C16jAwMAACtgGAHEBVuH7Vqm/qXXcoDJYqmblzVa8T8DC4/\nkyvI4vIzuVRzCwYpAQAAaBqCHUCE2d3UFxs0X2/SODynud7K0OScjoFB3X2FnQMZiTzVokuy\nAAAALYJgBxAxtQ562TrNis1qj/+veU3JioX5gcHdfQO6+o1a9JADAICoQLADiIDKOuazXzWr\ntqoCfKNIp1WJA7v5B3f3n5vvP4tbVgEAAFoFwQ4gLGXVzP9+0q79Q8U3vs/BrBMuHuyZNMir\nVSHPAQBAG0GwAzhLoki+3Kh5e6X2L610Zr0wZYhn8iCvSoFIBwAAbQrBDuBsVNYxLyzV7y5S\nnPpgRiI/fbhnVB+vAh8sAACQAn7/ALTaqm3q177VeXwnG+o6pPIzRrhH9PLRZz8fGAAAQLgQ\n7ABaweqi5y/Xb9ynbHiEocnUoZ6rR7tOOyUrAABAW4qtYCe6S7Zt+L3YzqR06T+gR7LydNv4\nq/Zs2Ly/mtNldB8woMDS8MtUdBzbtm1XiZVJ6txnQM+0034tQDjW7VEu+EJvd59slMtN4e+d\n5sjP5CSsCgAAoEEMBTuxet0L97+4hc7vnsEf+/DdzybOee6aXprG27h3Lb7vsS/rkgo6GO2f\nL37n8xmPP3l5VzUhYvkPTz78yl5Ffl6Cq/i9t/437v+ev/lcvTTHATLk8lKLvtH/8Luq4RGa\nIpcM8Vwzxq3AtK0AABAzYifY+ba8/9pG47SXnr8yT0ncf7x62+xXl5+/6PLcUzYRi5e/9oVn\n8P8tumuQiRJtv82/8+mFXw1bOD3bu/79N3ekXfPqk5NTGdG+cd6tz7y2bPRbszpLdjAgJ7UO\n+j/vGo9UnPywpFqEu6c4eucFJKwKAADg72Kmpzf/xy8bHL0mTM5TEkKItvcFgxLK1q8vbrTN\nie3bSpJHTBpkogghlGnA+CGWYxt/Kyek9NAhT07heakMIYQyDhxJpRyPAAAgAElEQVTYk64q\nLvJIcBQgLy6X6+Ax/79fM5+a6sYWehfdXodUBwAAMShmgl3JocO+5E6dDPXreXl5pKykpNFV\nLrfbTTSak1dnWZYlZaVlhJjNZqry6FE3IYQQ/s/DRwVjaupfruICtMLGjRtHjRrV6ZzJ/3xO\nrLKGPiZGrfjIlfa7pzgx5jAAAMSmmLkUa7fbidFobFhXGgxKodrmJKQh65H0jh1Vy3btrJme\nmUgIEUq27zhBvG6XQJLGXTtzw9MvP/Ds/oEZ7p1rfgoMvu2SXif37ff7x48f37B62WWXXX/9\n9dE+IIqiEhISov0qEqIoihBiNpulLiTydu/ePWPGjGvueNXgutrrD6U6i5579S6mc6ah+a+N\nRxqNRq1WS11FFFEUlZiYKHUVURT8MMr+GBUKxZm3i3MGgwzPMH8h77MNIUSpVEb7wygIQjPP\nxkywI4QQUTy1HYSiCNN4BAnVoJmXdb7vzQdmHx6Sr6v9Y+1uRyKj1upoIjisdp7wrqqSowG/\nPUBEb9UJB0k55QNy6qdFpVI1/02JCIZh2uBVJETTNEVRoig2/rHJwZw5cyZcsWC7/RqODz2i\nFMpS6+bmpT8vsx8pRVHBH6KM36vB0CPjAySEMAxD5H6MNE3L8mzToOHDKPtjlPcblWGYNjij\nNv8miZlgZzKbiNPpJCQ1uB5wOv2syahttBGVO+WpBTkrv9/6Z7Ur4fx7Hwx8fO8PycnEuvrl\nl1ZSM+a+PqOjghDRvuW/dz7x0ifnvHN9t+CXKZXKL7/88tQdVVdXR/uALBaL1WqV8UdUr9er\n1Wqbzcbz/Jm3jisHrf20KdeJ9YfVJYs7R/vNpx/9VFdXJ2ldkadQKEwmk9frdblcUtcSLTRN\nG41Gq9UqdSFRZLFYaJqW3/vzVEaj0e12c5xshxbSaDQ6nc7pdPr9fqlriRaVSsWyrLzPNgkJ\nCYFAwG63R/WFGIaxWCxNPRszwS4tt4Oqquioi3TSEUIIKS4uFrtMKmi0DVex+7cSXa+JV59L\nCCHEt+G5w4ael+WSkpVHueQJ53UMNtNTxnP7d2fWlpZ6STeZN/hCpIki+fBHraZgdkMeLyzg\n/3OZ7bVXjplMJikrAwAAaIGYuXlCde6w/urfV3xzjCOEiCd+Wrq2uuuQAYmEEMHnsts9HCGE\nKV/z8pOvrj4uEEKI/+gXX2y2jBrdgyLpmRlU1bYNh32EEEJE547tB/jMTnlIddAqvEDmfm74\n4MeTzcTjz+Oev8FTdbzo9ddfnzJlioS1AQAAtETMtNgR5aCrbyq8f8F9t2wvMFsPH7R3mPXU\n+BRCCCn5/IHb/5f1wFf/N4TqO21Wn/vfuP/ufeekew5vO6ie/OjMfIaQpLE3XL35qQ/uv+G3\nLh0TSNXhfXXJUx6e2knqI4J4IgjkxaWGtX+cHIK47sCinRWrrviCW7NmzaWXXjpz5kwJywMA\nAGgJKqY6gYmO4s0bfi9xKdN7Dh5YYA7eOWHbtWLFbuPQy4ZmE0KIaD+yZfOeUged1HXA4G5J\nJ4Opv2rvlh1/Vjppc1bXwnM7mZpti0Qfu/AF+9jV1dXJoI9dgCPPfGLcsDc0ER1FkevHu1LJ\n6i1btjAMM3DgwP79+0tbYZQE+9h5PB5593ppJ33sampqpC4kitpJHzu73Y4+dvEr2MfO7/dL\n28cutoJdm0GwC59sgl2Ap578yPDb/lCqo2ly9xTH6L4+QohGoxFF0ev1SlpgFCHYyQOCnQwg\n2MlAjAS72LkUCyCBAE899b9Gqe6eqY5RfXzSVgUAAHB2EOyg/fJz1KPvG7cfDo16StPk3qmO\nC5DqAAAgbiHYQTvlC1CPfmD8vT7VsQz5vxn2oT1kexEEAADaAwQ7aI+8furRD4w7/gylOgVL\nHpxhH9wdqQ4AAOIbgh20O14/NWeJcefRk6nu4Zn2gd2Q6gAAIO4h2EH74vFRD71r3HcslOqU\nrPjIlY7CfKQ6AACQAwQ7aEd8AWrO+ydTnUohPnqVvW+ngLRVAQAARAqCHbQXAZ56/EPDrvor\nsGql+PjV9nM6ItUBAIB8xMxcsQDRxPHkqf8Zth0KjVenZMVHr0SqAwAAuUGwA/kLzgO7aV8o\n1SlY8vBljj64AgsAALKDYAcyJ4pkwZf6n3aqgqs0Te6b5hjQFXdLAACADCHYgZyJInnla/3K\nrergKk2R+6Y5hvfC3BIAACBPCHYgZ++s0n3zWyjVURS5bbLz/N5IdQAAIFsIdiBb736v/exX\nTXCZosgtFzn/cZ5X2pIAAACiCsEO5OmTnzUf/6xtWL1unGviQKQ6AACQOQQ7kKHvt6vfXa1r\nWL16tHvaMI+E9QAAALQNBDuQm9/2K+cv14tiaHXGCM/l57slrQgAAKCNINiBrOwvYZ/5xMAL\nodUJ53mvHeuStCIAAIC2g2AH8lFcxcxeYvL6qeDqwK7+2yY5pS0JAACgLSHYgUxU2+k5S0wO\ndyjVdcvhHpzpYPAGBwCA9gS/90AOXF5q9nvGyrrQ+zk3lX/8KptKITb/VQAAADKDYAdxz89R\nc5YYj1awwdUkk/DE1TaDFqkOAADaHQQ7iG+CQJ7/1LCnWBFcNWrFp6+xpZiF5r8KAABAlhDs\nII6JIpn/hX7dHmVwVaUQH73KlpPCS1sVAACAVBDsII59/LN21bbQVLA0TR6Y7uiew0lbEgAA\ngIQQ7CBe/bJLteSH0KRhFEXuusQ5uLtf2pIAAACkhWAHcelAKfvi0pPTS8wa7R7TD1PBAgBA\ne4dgB/Gn2k4/8aHRz4WGrBvd1zdzJCYNAwAAQLCDeOPxUXPeM1bbQ2/dHrmBOy/B9BIAAACE\nINhBfBFE8txnhiP1Q9alWvg5VzgUDIasAwAAIATBDuLL69/qNu0LDW6iVYmPX2U36TBkHQAA\nQAiCHcSNlVvVX27UBJdZhsy+3J6biiHrAAAATkKwg/iw7ZDi5S/1Dav/usjZt3NAwnoAAABi\nEIIdxIFjVcwznxj5+ouu04Z5LuyPwU0AAAD+CsEOYp3DTc1eYnR6QoObDOrm/+c4l7QlAQAA\nxCYEO4hpgkCe/dRYWccEVzulc/dPd9CUtEUBAADEKAQ7iGlL1ui2HVIEly164bGr7BolBjcB\nAAA4PUoU2+OvSZ6P+t2UDMO0watIiKZpiqIEQYjeW+iXndT9r9PB3bMMWXiH0De/Td+uFEUR\nQmT8GaEoiqZpURQFQc6jxtA0Le8DZBiGtMlpTULBN6rsP4xRPaNKjqKo4G8NqQuJIoZh2uCM\nKooiy7JNPdvkE/JWV1cX7ZewWCxWq1XGH1G9Xq9Wq202W5R+nZRWM48sNjd8/26c4OyQ5I3+\nz60RjUYjiqLXK9sbNRQKhclk8nq9Lpdsuy3SNG00Gq1Wq9SFRJHFYqFpug1OaxIyGo1ut5vj\nOKkLiRaNRqPT6ZxOp9/vl7qWaFGpVCzLyvtsk5CQEAgE7HZ7VF+IYRiLxdLUs+002EH4jh07\n5nK5kpOT1Wp1xHfu8VFPfGh0+0Kd6S7o45s0SLbpCgAAIFLQxw5abf/+/ePHj+/SpcugQYM6\nd+781FNPRbbZWRTJi8v0xVWhGyby0rg7L8ZssAAAAGeGYAet43A4rrzyyoKCgtLSUrfbvXTp\n0s8++2zBggURfImP1mrX71EFlw1acc6VDpVCtle0AQAAIgjBDlpn2bJlarV63rx5iYmJNE0P\nGTLkhRdeWLBgQSAQmXkgth1SfLhWG1ymKfLAdEeaRc5dwgEAACIIwQ5ap7i4uHfv3sG78IIK\nCwsdDkdNTU34O6+0Ms99amy4rnvtWNe5+bLtRwwAABBxCHbQOsnJyUVFRac+UlRUpFQqm7lD\np4W8fuqRJQa7O3TDxODu/mnDPGHuEwAAoF1BsIPWmTx58v79+994443gSC4nTpx48MEHp0+f\nrlKpwtzzy1/qiypDt2nnpvD3XeqgMMMEAABAayDYQetkZma+9tprL730Urdu3QYNGtS3b1+z\n2fzkk0+GudvV29VrdoSioU4tzrkCM0wAAAC0Gsaxg1YbM2bM5s2bt23bZrPZOnbs2KdPnzB3\neKyKeeVrXXCZosg9Ux2ZSbhhAgAAoNXCDHa8w+4zGLWRqQXih8lkmjRpklqtrqurC3PmiQBH\nPfupwesPXXadPMgzuDtumAAAADgbYV6KrXvv4pS84Vc8tOibnSciM9oFtDeLvtUdOR76A6Nj\nGnfdOLe09QAAAMSvMIOdpsvAPuKOj5+5ZWLvjNTu465/4t01h2xynt8XImvdHtWKzaEZyTRK\n8eHLHAoWXesAAADOUpjBTjfm6XVHT5Rv+2rRw1eeR7YtmXPt6ILUzPOm3jV/+fYqXE+DZlXU\nMfM+1zes3jbZmYWudQAAAGGIxF2xqtR+E29+cvGqvZVV+9csefr6geS31/89pTAz45zJd87/\nZm8dWvDg7ziePP+pweUNda0bf653VB+ftCUBAADEu4gOd8KYu4y8ePqMmTMvHZWvJVzNrq8W\n/Htij6z8cf/+YLcrki8E8e/tVbq9x06OWvevi/AOAQAACFeEhjvxVf6x5qvly79Y/tUPO6v8\nRJHYa8xNT82cOf2CtMNfvvnScy9fNcaRcuTtsZrIvBrEu60HlV9sCL0bFKz4wAyHSoGudQAA\nAOEKM9gF/vzu5VfeW758xYYih0BoQ+fhMx5+/LLLpo3pkRjac+dbXhrbxZ08euXavWRsYfgF\nQ9yrttHPf6YX64PcbZNceWmcpBUBAADIRJjBzvbdc/fM+1mT2f+Suy+7bOaMC89LV/99Izqh\n59jJpGdCeC8FsiAI5NlPDXZ3qA/AyHN84wq90pYEAAAgG2EGO83A25f89M7Fw/MMzc3q2ffW\n/30R3uuAXHz6i3Z3kSK4nJHI33GxU9p6AAAA5CTc4U7OnXpVz+oPb/9Hv5wZS4K9310fXdVp\n0LSHPtqN39jwF38eZz9cG5qnRMGID85waFXoWgcAABAxYd8VW/zqJSP/9covdZ165gZb/9js\n/NTSFc9cMWjSq3+GXR7IR4CnXvhMH6jvTXf1aHd+JrrWAQAARFK4wW7zKy/+Soa8uGX/2tkj\nVIQQQlTD5mw4uGPhBNXaOU+uwhjFUG/xKm1RZejSf/ecwNShHmnrAQAAkJ8wg513374icu70\nq7qpGj2sKbh19qyMms2b0WYHhBBC9hQrvtgYGt9ErRTvneakIzqEIgAAAJCwgx3DshRxOBx/\nf8bn8xOnE/3sgBC3j3r+M4NQPwPJzRe6MhIxdRgAAEDkhRnsFCPGjFDsWHjfm/vcpz7s3bvg\nmY+rdf379whv9yALi77RVdaF3mmF+X6MbwIAABAl4c48kXX93DnvDZt9Y+8OC0aOHdYz28I6\njx9Y/+13v1dphs977BJtRIqEOLZxn3L19tDohkatcO80J9Xc0DgAAABw9sKeUkzZ5z9rNmbP\nvvfpxT98uGg1IYQQStdh5B3vzX/66u7oRtXO2Vz0/C/0Dau3T3ZZ9EIz2wMAAEA4IjFXrL7n\nrHkrZ73oOn7k6HGbX5nYoaBjglIMeJyegE6jQPNMO7bgS53VGYr3o/v6hvX0SVsPAACAvEUi\n2AUxuvT8nukNq1VvjEubO2z7n0/1jdgrQHz5fpt6/Z7Q7dJJRuGmC3ErDQAAQHSFf7G0csUD\nozslaBRsY+m3/UoZDPozfz3IUqWVee1bXXCZoshdU5wGDSaZAAAAiK5wg51v5cNXP/+Tt2DS\nFVMLEwWm86grr5x58dA8raDue+vnS+/Kj0iREGdEkcxdpnf7QpfhJw/yFOZjrGoAAICoC/dS\n7M+ff17T+a6t618oZMpeOpa1eNC97z7akwSOLJp43sLNdczFESkS4szKreo/jiiCyzkp/D/H\nuZvfHgAAACIizBY7d2lpHVs4oB9DCMns3Fmz7/fffYQQRd7Nj17rePmlbzBrVPtT66DfXhW6\nCEvT5N6pDiWLi7AAAABtIcxgp9LrFVxtbXDmidzcHOHgwcOEEEKonj27Ozdv3hdufRB3Xv1a\n7/SELsJOGeIpyOKkrQcAAKD9CHdKsYFDBjBr5z/09REnT7r26aM68PXSXX5CiGvTpl1ErVZH\npEiIG+v3qNbtUQaXU838lRfgIiwAAEDbCffmiZxrH7kh5+ArkzpNfK1SPfbKaUl/PDa858hx\nw8+55PWKrpMvKohIkRAfXF5q0Tcn74S98xKnWomLsAAAAG0n7HHsjKMXbd910Wdf1/TSEeNF\n85c/57ll3qpfKpMKb3zvzTn9IjdMHsS+N1boqu2hPxXG9PP26xyQth4AAID2Jszk5asuKrGy\naeOuvy+4o8Qh9y/74/6z3p3oLtm24fdiO5PSpf+AHsnK023jr9qzYfP+ak6X0X3AgAILc/KL\nXUVbNuwssRFTh76DC7O1mPKiTe06qvi+fk5Yi164YbxL2noAAADaoTAvxXo/vzE/f+qbxyNR\nili97oXbbn/u41937ljzzpxb7n1319/vqXXvWnzXLQ+/8c1v2zd8sfC+mx74aL83+ARf+s2j\nN901/6vtB/b+9MEzd9z52k5vJGqClvFz1LzlerH+uuutk1wGLS7CAgAAtLUwg53pgnEDFXt+\n+ulE+JX4trz/2kbjtOdefeGxx+e+Mud85+evLi9uvIlYvPy1LzyDH1i06Pmnn1+06KHB1Z8s\n/KqEEEJOrHjl7cNd75y/4NGH5sxdcPcA98qP19SEXxO00Ps/aMtrQo2n/bv4h/bAnLAAAAAS\nCPfmic53vP/a+D13j79+0crth0srGqtytHy6Af6PXzY4ek2YnKckhBBt7wsGJZStX9842Z3Y\nvq0kecSkQSaKEEKZBowfYjm28bdyQirWrtmTNHr6iCSKEEKMA6+6+19j8sKfLQ1a5EgFu3yD\nJrisVYl3TMacsAAAANIIs49d9cIx+bf/TAh5+5YJb//t2cRb11YvHNmyPZUcOuxLHtTJUL+e\nl5dHvi0pEUnuyb5ybrebaDSahnWWZUlZaRnxOvcfVfW6Mqt82w9bipzqlILCgePPV5z1QUEr\n8AKZ97me40Or109wJZkESSsCAABov8IMdtrCyx94YGBTz+oG57R4T3a7nRiNxoZ1pcGgFKpt\nTkIash5J79hRtWzXzprpmYmEEKFk+44TxOt2CXU1NaLGvvTueyrUnbKVNUfee/uj0Q88f+u5\n9V/p9/uvu+66ht1MnDhxypQpLT/Is0PTtMlkivarSIimaULIV7+ZDpWFGkf7dhYvG62mKPmM\nXhg8RhmPx0hRFCFEpVIpFHL+S4hhGLPZLHUVUUTTNEVR8j5GhmEMBoMoyrbzbvBso9PptFqt\n1LVEC0VRFEXJ+2xDCFEoFNH+MDb/QQg32A268dlB4e3iFI1LpSjCsMypz6sGzbys831vPjD7\n8JB8Xe0fa3c7Ehm1Vkd7Sj3EWk4mvfTGtBwlEevWPXfH8y9/NOK9m3o2fGlZWVnDss1mY5hG\nO44GiqLa4FUkRFHUsSry9opQqlMqyH+uIiwrw0OW98+R1McCqauIovbwYSRyf6NSFBWMPvLW\nHo5R3m9U0iYnHEFo7spYmMFOcJ0or2uyozyjT043q1q2J5PZRJxOJyGpwfWA0+lnTcbGf7lQ\nuVOeWpCz8vutf1a7Es6/98HAx/f+kJxMVEdVhOo99qIcJSGEUJah4wYu2nD4cA3pmUgIIUSp\nVP7444+n7qi6urrlB3l2LBaL1WqV8d+Xer3++f+pffVj1V0+0qVjPTXyumVFo9GIouj1yvYW\na4VCYTKZPB6PyyXb4WlomjYajVarVepCoshisdA0XSOzj19jRqPR7XZznGynKNRoNDqdzuFw\n+P0t75weZ1QqFcuy8j7bJCQk+P1+u90e1RdiGMZisTT1bJjBrnbxpdm3/9zUs63pY5eW20FV\nVXTURToFpy4oLi4Wu0xqPHMFV7H7txJdr4lXn0sIIcS34bnDhp6X5ZLUskxW9J8SoURBIGq1\nhkAU/fQHs2lvaDkvjZs27O+j0wAAAECbCvdSbP9rHnlk5Ml13ueoKd27btUPu8nQux6+dkrL\npxRTnTusv/qFFd8cGzkjhxVP/LR0bXXX6QMSCSGCz+X00VqjhmXK17z8ZOmlrz03NZ0m/qNf\nfLHZMurZHhRh+g4+TzN/1Tclwy7NVhLRtu773/w9ZvSVbT+FGOALUAu/CL15KIrcNtkpx2uw\nAAAAcSb8YPdo/78/7Pz9yfHD52+89T93tnxXykFX31R4/4L7btleYLYePmjvMOup8SmEEFLy\n+QO3/y/rga/+bwjVd9qsPve/cf/d+85J9xzedlA9+dGZ+QwhRDvshju3PvziPf/6rXuupubQ\n7tqcWU//IzW8Q4PmfPCjtqI21CtrTD9v9xzZXh8BAACII1SUOoFxX88yTzr8TPn629Nb8VWi\no3jzht9LXMr0noMHFpiDbUC2XStW7DYOvWxoNiGEiPYjWzbvKXXQSV0HDO6WdEow9VX+sf63\nQ7W8Iav3oP55xmZ7oKKPXTjKqpmbF5gDPEUIMWjEt+6qM+nkOcQJ+tjJAPrYyUM76WNnt9vR\nxy5+yaOPXZNolqVIcXExIa0JdpQhd8C43AGNHzT1+sdlvU5uYszrPzrvNK2ERJXa+4JJvc+q\nWmiVl7/SB1MdIeSf4z1yTXUAAABxJ9xgF/A4ffxfHuOdpevnP7HMSfXskBvm7iHm/LRTtePP\n0ChE3XLJP87zybFREgAAIC6FO/PE6xOSm7grlsm99oErWtNcB7HP46Pe+i542zKhKXL/TELT\nhP9rsgcAAABphBns1D0uvO66zo0foxi1MbVg8NQrp/Ru8gowxKf312ir7aHeixMH8b3ymLo6\naSsCAACAk8IMdvrz73vr/MhUArGuuJL5alNocECDVrzxIo4QjHECAAAQQyIwe0nN5tdu+0e/\nnBlLgje6uD66qtOgaQ99tNsZ/q4hZogieeVrPVd/1fW6cS6TDn3rAAAAYkvYwa741UtG/uuV\nX+o69cwNtv6x2fmppSueuWLQpFf/DLs8iBU/7lDtPBq6Z6IgkxtbKNsRQAAAAOJXuMFu8ysv\n/kqGvLhl/9rZI4KzwqqGzdlwcMfCCaq1c55cJdvheNoXt496e9XJeyZuneSk5TxlPAAAQLwK\nM9h59+0rIudOv6qbqtHDmoJbZ8/KqNm8GW12svDu99paR+itctEAb5cs2Y4RCgAAENfCDHYM\ny1LE4XD8/Rmfz0+cTvSzi39Flew3m0P3TJh1wlWjZTtoOAAAQLwLM9gpRowZodix8L4397lP\nfdi7d8EzH1fr+vfvEd7uIQa8/q1OqJ9a4rrxboMG90wAAADEqHBnnsi6fu6c94bNvrF3hwUj\nxw7rmW1hnccPrP/2u9+rNMPnPXaJNiJFgmQ27FX+Xj/PRH4mN6ov7pkAAACIXWHPFavs8581\nG7Nn3/v04h8+XLSaEEIIpesw8o735j99dfcIDKYC0glw5O2VoXsmKIrcfKEL90wAAADEsrCD\nHSFE33PWvJWzXnQdP3L0uM2vTOxQ0DFBGYH9gsS+2KApqwkNQXxBb1+P3IC09QAAAEDzIjdA\n8eXLjPk9+53br+PGO7thgOL4Z3XRH/8cupSuZMVZY3DPBAAAQKzDAMVweotXaV3e0JXXGSM8\nKWah+e0BAABAchigGE7jyHF29e/q4HKSSZg61CNtPQAAANASGKAYTmPRKUOc3DDepVZiiBMA\nAIA4gAGK4a9+3a3aVT8tbPccbngvn7T1AAAAQAthgGJoJMBRb30XumeCpshNFzopDHECAAAQ\nJzBAMTSybJ2m0hoa4mRUX0wLCwAAEE8wQDGcZHXSn/4SmhZWoxSvGeNufnsAAACIKVEboFgM\neJyegE6jwIW8+PHWSp3b1zDEiTvRiCFOAAAA4kkkgl0Qo0vP75nesFr1xri0ucO2//lU34i9\nAkTV4XL2xx2hu5tTLfzUoZgWFgAAIM6Ef7G0csUDozslaBRsY+m3/UoZDPoIlAht47VvdUL9\nqCY3THArWAxxAgAAEGfCDXa+lQ9f/fxP3oJJV0wtTBSYzqOuvHLmxUPztIK6762fL70rPyJF\nQtRt3KfcXRQa4qRXx8DQHhjiBAAAIP6Eeyn2588/r+l819b1LxQyZS8dy1o86N53H+1JAkcW\nTTxv4eY65uKIFAnRJQjk3e91wWWKIjdOwLSwAAAAcSnMFjt3aWkdWzigH0MIyezcWbPv9999\nhBBF3s2PXut4+aVvMBNVPFixRV1cFRri5PzevvxMDHECAAAQl8IMdiq9XsHV1gZnnsjNzREO\nHjxMCCGE6tmzu3Pz5n3h1gfR5vFTH/4YGm9QwZKrRmGIEwAAgHgV7pRiA4cMYNbOf+jrI06e\ndO3TR3Xg66W7/IQQ16ZNu4harY5IkRBFy37V1DlDb4NJAz3pCby09QAAAMBZC/fmiZxrH7kh\n5+ArkzpNfK1SPfbKaUl/PDa858hxw8+55PWKrpMvKohIkRAtVhe9bH1oRGK9RpwxAs11AAAA\ncSzsceyMoxdt33XRZ1/X9NIR40Xzlz/nuWXeql8qkwpvfO/NOf0iN0weRMOS1VpP/YjEM0e4\njVoMcQIAABDHIpG8TF0vvL5rcDFxyP3L/rg/AvuE6CutZlZtC10sT7UIkwdhRGIAAID4htlc\n26+3Vur4+jnDrh7twojEAAAA8Q7Brp3aXaTYtE8ZXM5L487vjRGJAQAA4h6CXXskiuTtVdqG\n1esnuGhKwnIAAAAgMhDs2qNfd6v2HQtNIHZugb9f54C09QAAAEBEINi1OxxP3lsdaq6jKXLt\nWAxxAgAAIBOUKLbHLvMcF/VZsxiG4flYHOz3k7X0vKWhQH/RQOE/VwnNb98UmqZpmuZ5XsZv\nIZqmCSGCcJbfothHURTDMIIgyPgYSQx/GCOFYRiKotrgtCah4BtV3mcb2Z9RKYqiKEreZxuW\nZUVRjPYJRxRFhULRZA1Rfe2YZbPZov0SZrPZbrfH2kfU5U2NjVEAACAASURBVKXe+c4cXFay\n4swRNpvtLD9jer1epVI5HA4Z/8rUaDSiKHq9sh0IhmVZk8nk9/tdLpfUtUQLTdMGg6ENPvIS\nMpvNNE3L+xgNBoPb7Zb32Uar1brdbr/fL3Ut0aJUKhUKhbzPNhaLJRAIOByOqL4QwzBms7mp\nZ9tpsGubvCWKYqwFu6W/amyuUHPdJUO8SUb+rAsMHloMHmMEifWkLiS65H2MDW9UqQuJOhxj\nXGsPZ1Qi9wNsOLRoH2Pz+0cfu3bE5qKXbwhNIGbUitOHo3cdAACArCDYtSMf/6xpmEBs+nC3\nTi3bP5sAAADaJwS79qLGTn/7W2gCsQSDMHGgbPuNAQAAtFsIdu3Fhz9q/Vyoue6KC9wqBZrr\nAAAA5AbBrl2orGNW/x5qrks182P7obkOAABAhhDs2oV3V2sD9UNcXT3arWinN0MDAADIHIKd\n/BVVMj/vVAWXc1P583v7pK0HAAAAogTBTv7eW60T6jvUXT3KReNnDgAAIFP4JS9zB0vZTfuV\nweX8TG5wd9mOaQ4AAAAIdjK3+HtdwwjV14xxUZSk1QAAAEA0IdjJ2a4ixe9/huYJ7tkhUJgf\nkLYeAAAAiCoEOzlbvErXsDxrDCYQAwAAkDkEO9natE+591hoXJP+Xfy9OqC5DgAAQOYQ7ORJ\nFMmSNdrgMkWhuQ4AAKBdQLCTp593qY4cDzXXDe/p65TONb89AAAAyACCnQzxAnn/h1BzHUOT\nq0ajuQ4AAKBdQLCToR93qMpqmODyqD7erCRe2noAAACgbSDYyQ0vkI9/CjXXsQy5YpRH2noA\nAACgzSDYyc2pzXVj+nlTzWiuAwAAaC8Q7GRFEMgnP59srps5As11AAAA7QiCnays2aEqrT6l\nuc6C5joAAIB2BMFOPtBcBwAA0M4h2MkHmusAAADaOQQ7mUBzHQAAACDYyQSa6wAAAADBTg7Q\nXAcAAAAEwU4e0FwHAAAABMFOBtBcBwAAAEEIdnEPzXUAAAAQhGAX39BcBwAAAA0Q7OIbmusA\nAACgAYJdHENzHQAAAJwKwS6OobkOAAAAToVgF6/+2lw3Es11AAAA7R2CXbxat+dkc92oPt5U\nM5rrAAAA2jsEu3j16S+a4AJNk+nD0VwHAAAAMRfseEfFkYOHjtV6xSY3Eb3VxQcPHC6z+U+/\nh5oju47Wyrz5avMB5eFyNrg8vKcvM0nmxwsAAAAtwUpdwCkCxd8+/8Q7v9UQhcgpssfePvuW\nISlU403EqnUvP/byDyVeihIpfeeL/j37uvMsp24TOPzxo/d9Yr7xgycmGNu0+Lb18U+h5jqK\nIjNwMywAAAAQQmKpxU488unzbxzocOMr/1u29KO50ww/z3t5de1ftrGtWbTgV/WEJ975bPln\nix8bQ6988eWfbKc87z/04bylx4S2LFsCO48q9h5TBJcHdPV3TOOkrQcAAABiROwEu72rVpbk\nTfrnuGw1obSdpkwdzO5cu76m0Sa2DWu2Kc+/+qreSUpamdj76iuG09t+3NCQ7Pz7Ppj3Y/b4\nIaY2r71tffyTtmF5BnrXAQAAQL2YCXbH9+23mbp1ywit0l275IsHDh5stE31iWqSmJLC1G9j\nMZvEw3/+GVzz7lny37XZN/1rmLxz3cFSdvvhUHNd306BbjkBaesBAACA2BEzfexqa2tJYmJi\nw7rOZGQ4m91NyMnmqZTUVKqypMRP0pWEEOI4cvQEcTucAiG0d/fi+T/n3bRwiKls99/2HQgE\nZs+e3bA6cuTI888/P5oHQwghNE0bDAZRbPoukLOydL2iYfnaCcRgMER2/y3HsiwhRKfTRfwY\nYwfDMIQQhUJxxi3jFE3ThBClUhlckCWKohiGkfCT0gZomqYoSt7HyLKsTqcTBNl2tQmebTQa\njUqlkrqWaGEYhqIoeZ9tCCEsy0r7YYyZYOcP+ENv7BClUkF4rtHdnoZhE0d8+Owbz2e4x+dr\na37//OPdFM0qFTTx7Hj7vxs63bRwqImQsr/vWxCEH374oWE1Kytr/PjxUTuSk5RKZWR3ePQ4\nWbcrtNyjAxncS/rAEfFjjEHBCCtjDMM0+uzJkYx/WTaQ/THKOBA0kPGfkQ1kf7ahaTraH8bm\n/8KJmd9YJqOJeDwnO4wJHo+PMhr1jTbSDrjt2bs/evfrz15bFTDmD//nFOULP1gs/N4PFvxo\nGP1AcsWBAxWkxMoTd+XhA4czszunBlv7lErljz/+2LAXlUpVU9O4914UmM1mm80W2dasRV/o\nBTH0dpk+3FFTc/oRX9qGXq9XqVRWq5XnZTvYikajEUXR6/VKXUi0sCxrMpm8Xq/L5ZK6lmgJ\ntp3bbLYzbxq3zGYzTdO1tX+93UxODAaDx+PhONneK6bRaLRarcPh8PulPLFHlUqlYllW3mcb\ni8Xi9/sdDkdUX4hhGLPZ3NSzMRPsklNTqcrSUo7kBEuqLD8uZg3IaTTcieh3e439r3po2DXB\n1V2vXKXsOj6Pq1nB6WrWLHxqDSGE8B4n4VfOe2rLqIdeuaYrIYQQiqKMxkajn0T7mx6qVxQj\nGOwq6pifd4ZSXW4Kf16BT9pLoMFDi+wxxhqxntSFRJe8j7HhjSp1IVEn+2NsJ29UeR+j7A/w\nLwvRfqHTiplgZxgwsOerH/66yT14qJaQwNG164tSBtye22gbfvP8WXPJvz96cISaEOLevHoj\nd96tfZQq7d1LhjVsdPid6+8+OvUV+Y1j9+nPGr6+8fXy89001ezWAAAA0P7ETLAjpgtm/OPL\nOQsfmls0KLl26+o1tqH3Te5MCCHHVzz91IrU6xde14ctHDfK/Nhbj71ZNyLTu+e7z7d2vPbF\ngdoz7VgWah306t9DzXXpCfzQnj5p6wEAAIAYFDvBjmjOueH5p7OWrti276iy88WP3DepX3Dg\nElqlMxo1LCGEaPr966Unc5ev2b15K5U47K4XJw/K/Fu7lSa9S0+SGEPHFRGf/aoJcKFjnTnS\nw8i/DzEAAAC0GiXjq93NqK6ujvZLWCwWq9UakW+v3U3NeiHB46cIIUkmYfHdtYoYCK56vV6t\nVtfV1eHmifilUChMJpPH45F3d2aj0Wi1WqUuJIosFgtN021wT5iEjEaj2+2W980TOp3Obrfj\n5on4RdN0QkKC3++32+1RfSGGYSwWS5NlRPW1ISKWr9cEUx0h5NJhnlhIdQAAABCDEOxindtH\nff2bJrhs1gnjz5Vt6xEAAACECcEu1n27We30hJrrLh7sUSna46VzAAAAaAkEu5gW4MiXG0LN\ndTq1OHEgmusAAACgSQh2MW3tTnW1PfQz+kd/r06N5joAAABoEoJd7BJFsmxdqLmOZcikgZ7m\ntwcAAIB2DsEudm0+oCyuDE2WfEFvb7KpuUl/AQAAABDsYtfSX0PNdRRFpg5Fcx0AAACcAYJd\njDpYyu4qUgSX+3fx56bKdhBgAAAAiBQEuxj1yS+ahuVpw9BcBwAAAGeGYBeLKuqYjftUweWC\nLK5Xh4C09QAAAEBcQLCLRZ/9ohHq75SYMRzNdQAAANAiCHYxx+aiV28PNdelWfhB3XzS1gMA\nAADxAsEu5ny1Se3nQnOIXTrcQ+NHBAAAAC2D1BBbfAHq603q4LJZJ4zph+Y6AAAAaCkEu9iy\napva7g79UCYN8ipZzCEGAAAALYVgF0MEgSxfH2quUynEiwZ4pa0HAAAA4guCXQz5dbfqeG1o\nDrFxhV6jFnOIAQAAQCsg2MWQz9eHBiWmaXLJEDTXAQAAQOsg2MWK3/9UHChlg8vDevrSEzCH\nGAAAALQOgl2sWLbulDnEhmJQYgAAAGg1BLuYUFzFbDukDC736RTIz+SkrQcAAADiEYJdTPhy\ng0asH9hkKprrAAAA4Kwg2EnP4abW7AjNIZaZxBfm+6WtBwAAAOIUgp30vtms8QVCc4hNHeKh\nKWnLAQAAgHiFYCcxjiffbg4NSmzQihf0wRxiAAAAcJYQ7CT2805VtS30U7jwPI9aiTnEAAAA\n4Cwh2Ensy02hUU5YhmAOMQAAAAgHgp2UdhUpDtYPSjy8ly/JhDnEAAAA4Owh2Elp+fqTgxJP\nHoRRTgAAACAsCHaSqbQym/aHBiXu2SHQJQuDEgMAAEBYEOwks3y9Wqi/9HrxYDTXAQAAQLgQ\n7KTh8VGrt4dGOUk184O7YVBiAAAACBeCnTS+26p2eUMjEV882Evj5wAAAABhQ6CQgCCQrzaG\nmus0KnFsIUY5AQAAgAhAsJPAxn2qijomuDy+0KtTY1BiAAAAiAAEOwksWxdqrqNpMmkQmusA\nAAAgMlipC5CGwWCI9kvQNK3X6//++P5j1N5jiuDy8F5CQa422pVECcuyhBCdTieKsm1xZBiG\nEKJQKKQuJFpomiaEKJVKWr7dPCmKYhimDT7yEqJpmqIoeR8jy7JarVb2ZxuNRqNSqaSuJVro\nelIXEi0URRFCWJaV9sPYToOdxxP14UUUCoXX6/37aejDH04mucmD3B5PvA5fp9VqGYbx+Xw8\nz0tdS7So1WpRFH0+n9SFRAvLsgqFguO4NvhESIWmaYZhZHyApP5vD3kfI/3/7d17XBTl/gfw\n78zel2UBQVYN4QheE6/kPUUlL3n3aPUqy9JjgqaWHLNSO0ZZXo9hapapx3tqaprXvF9SvGuo\nCKQIooAELsvC7rK7s/P7gwWxX3UO5OzA+Hn/tXPZne+zC7OffWbmGZaV9t5GpVLJ5XK73e5w\nOMSuRSgKhUIul0v4D5VhGKVSyXGc0G1kWfZPfgA8ocHO6RQ8TvE873Q6fxPs8gvZk9fcgxI3\nrOdsVt8mfCFCcblcROR0OiW8q+U4rvRzFLsQoZT+vnS5XBJuI8uy0v4QiYjneYZhJN9GjuMk\n3MbSdC7tNspkMsnvbajs21/QDZX27/5hGYJuG35j11m1o+zjHopBiQEAAOCxQrDzHAfH7D/v\nvmzCX+/q1kKyB/gAAABAFAh2nnM8UVlQ7H7D+7e3KZ7Qw+AAAAAgFAQ7z9l1RlP6QCGn59th\nlBMAAAB4zBDsPCQ5U55y191H1zW8xE/nErceAAAAkB4EOw/ZdVZT/nhAB1w2AQAAAI8fgp0n\nmIrZE1fdo5yE1XU+HSzZi70BAABARAh2nrD3vNrhZEofD+mMs+sAAABAEAh2guNctOece5QT\nvdbVvSVGOQEAAABBINgJLuGGKs/kfp+fb1eikEv2XocAAAAgLgQ7we1McHfXsSz1a4/jsAAA\nACAUBDthZdyXXUtXlD7u1Mxu8JXsbVUBAABAdAh2wtp5RsOXHXodiFFOAAAAQEgIdgIqsjJH\nrqhKHwcHcq1CHeLWAwAAANKGYCegHy+qbHb3KCcDO1gZRtxyAAAAQOIQ7ITC8w9HOdGo+Kg2\nGOUEAAAAhCUXuwDJSkhi7uW5c3OfCJtWhVFOAAAAQFjosRPKlmPuI68MQwM6YJQTAAAAEByC\nnSCyH8jOJrmDXduG9qAAjHICAAAAgkOwE8QPCWpX2aHXQR3RXQcAAACegGD3+PE8/ZzmHpTY\n4Me1a2IXtx4AAAB4QuDiicePYWjJhIK03FrrDjhbNnCwGOUEAAAAPALBThAsQ+2a8o3qmHke\nF8MCAACAh+BQLAAAAIBEINgBAAAASASCHQAAAIBEINgBAAAASASCHQAAAIBEINgBAAAASASC\nHQAAAIBEINgBAAAASASCHQAAAIBEINgBAAAASASCHQAAAIBEINgBAAAASASCHQAAAIBEVLdg\nx5lz0lJ/ufPAxv/hKrwtLyM15eY9k/23zy26fzs1OfVPnwwAAAAgWXKxC6jAkbFn3ierzuaT\ngncq6vee+OH4LoHMo6vwuT8tjlt8KNPGMDyjazjgnQ//0c6PIaLipG9nz92SWEAy4jhV/Z7j\nZ77d/bdP9hSLxbJz587s7OxatWoNGDAgICBAnDoAAADgCVN9euz4tC3zlqf8bezSb7dt3bhw\nuPfxzxcffPCbdUyHl31xUv38J6u++/67/8T1YvcvWHzMRERc4tp5m7LDJy39dvv2LV9/EOk6\n9uWyw0YxWkGpqamdO3eOj49PS0tbv359x44djx49KkolAAAA8KSpPsEu6cf9maGDRvepryZG\nG/b3YZ3liUdP5T+yiun04YvKHiNfaxWgZJX+rUaO6MZePHLaRJSSkPCg6eBRPeprGUZVt9Po\noa1LLiVc/u2hWuHxPB8dHd2zZ8/Tp09v2LDh8OHDU6ZMGTdunNEoTsoEAACAJ0q1CXbZN5JN\nPs2a1XNPsk2bNOJTUlMfWSfv1zzyDwyUla3j5+vD37x1i3hZ3dZRPcPLj3jyREQc5/JI4RWl\npKQkJyd//PHHMpm7yJiYGJ1Od/z4cY/XAgAAAE+canOO3YMHD8jf37982stHL3OaCi1E2vJ5\ngQYDcz8z0051lURE5rTbv5LFXORi2g6KbVK2Em+6tHrbZfnTb7ZUl81yOp3Lli0rf5mIiIg2\nbdoI0Qi73a7RaAwGAxGxLKvVaomobt26VqvVy8tLiC2KSC6XE5FWq3W5PB+hPUQul/M8Xx7T\npYdlWSJSKBTS+/ssxzAMy7ISbiCVfY7SbqNMJtNoNNLe2xCRWq1WKBRi1yIUmUwm7X9GhmGI\nSCaTidvGahPs7A47PfIFqlQqiHNyFdfx7jowcsOc5fPqWfo20uZf3r7pGsPKlYqHvY6O3HOb\nl3y5LZFtN37C84by2RzHrVmzpnySYZjOnTsL0YiWLVtaLJbExMQOHToQkUajuX///tWrV1u3\nbq3RaITYouhUKpXYJcBfJZfLS79UJEyq/4AVSb6NEv6JVU6pVIpdguAkv7cp/REi6Cb+/BdO\ntXl/ffQ+ZLVay6ddVmsJo9frHllJ22HCnNiNq3d999WPDn2jbqP/rpx/yM+vdJklbe+Sef85\nVVAvauz80X0b6SpcEqtQKNatW1c+6e/vX1BQIEQj5HL5xIkTX3jhhblz50ZGRiYmJk6fPr1r\n167h4eECbVFEWq1WqVSazWaO4/772jWTSqXied5u9/zpmh4il8t1Ol1JSUnF/z2JKe0hMJvN\nYhciIL1ezzCMyWQSuxABeXl52Ww2ae9tNBpNcXGxw+EQuxahKJVKmUwm4b0NwzA+Pj4Oh6O4\nuFjQDbEsq9fr/2hptQl2tQ0G5v7du04KLi3pflY2H9Qh+JERS3i7xaZv/9q0rm+UTl5d+pqy\nad9QIrKnbHh/+raSdiPnRA9q5vvbEwdZlm3WrFnFOXl5eQK1Y+rUqRqNZty4cSaTSa1Wv/zy\nyzNmzJDkzqj0F4PT6ZRk60opFAqe551Op9iFCKX0wIHL5ZJwG1mWlfaHSEQ8zzMMI/k2chwn\n4TaWHoGVdhtlMpm0/1BLT4rwwA7nz3uvq83FE94dOoZzl0+esRARkeP20VPpgR3ahzyyDndu\n0esjF/5kK52ynDuY4Gz3bGslUdbOZVtM7SbPfW/I/091HqZQKGJjY2/evHnv3r2MjIx58+b9\nSawGAAAAeIyqTY8d+fR8qd/Ofy2ZtjC9U+0HFw4eNj377uCGRETZez/7dK9hzJJ/tJZH9Iny\njVsR940x8inb9X3bLzQYtaCjlijv/Nk0Re3QrB83b374eoERQ3s0FPFsBYPBUFBQwPO4CwYA\nAAB4SPUJdqRp+ea8z4K27r1447ay4ZCZ7w5q60NERKzKS6/XyImING3H/XtWyPeHr527wPh3\nnbxgcKenGCIys37hjYtzfv45p8LLNQgeKGqwAwAAAPAw5snsUhLuHLtyfn5+0u6x0+l0arXa\naDRK+Bw7jUbD87zNZhO7EKEoFAofHx+r1Sr0qb4iKj3LWHpXL1Xk5+fHsmx+fv5/X7XG0uv1\nFotFwqdnaTQaLy+vwsJCCV+tpVKp5HK5tPc2tWrVstvthYWFgm5IJpP5lV05+jtlCLptAAAA\nAPAYBDsAAAAAiUCwAwAAAJAIBDsAAAAAiUCwAwAAAJAIBDsAAAAAiUCwAwAAAJAIBDsAAAAA\niUCwAwAAAJCIanRLMU9SKBRCb4LjOLlcym9venq60WgMCwtTqVRi1yIUhmEYhvHAX4tYrFZr\nUlKSv7+/wWAQuxahMAzjcrkk/CES0ZUrVziOa9KkidiFCIjneZlMxjCM2IUIJScnJzc3Nygo\nSK/Xi12LUFiWJY98/4rF6XQmJCR4e3vXr19f0A2VvpN/5Am9pRj8dbNmzdqxY8fmzZvDwsLE\nrgWqKDExcfTo0SNGjJg8ebLYtUDVDRs2zGg0HjlyROxCoOrWrFmzePHi+fPn9+jRQ+xaoIqM\nRmOvXr26deu2cOFCEcvAoVgAAAAAiUCwAwAAAJAIBDsAAAAAicA5dlBFSUlJWVlZHTt21Ol0\nYtcCVWQymc6fPx8cHNy4cWOxa4GqO336tMPhiIyMFLsQqLr09PSbN2+2bNkyMDBQ7Fqgiux2\n+4kTJ2rXrt2qVSsRy0CwAwAAAJAIHIoFAAAAkAgEOwAAAACJkPIIuiAU3nh5y/J1B67cMTrU\ngU06Dx875rkQpdhFQVU5b66PffdA80/XRj8tdilQecUpu79ZufN82gOH11Ot+r054aUWPmKX\nBJXF/Xph09frj17PKtIYmnZ7JXpEp7qSHcNXivjry9/4pCR208TyE+u4nITVy9YdvZHr0AVH\n9BsVM6yF3oNDa6PHDirt3vefztp6t8Hw2LhPpo2MKDnxRdx/rtnFLgqqyJ668fOt6ZzYZUDV\n5P44b+aatAYvT5s/d/oLQenfzl1xzip2TVBJ3O0tH8/aX9L+9ff/9f6INpb9cz5al4w9ak3h\nsmRfWrf2sLHiPC51Xdzco87O42Z+OmNk89zv4mbvyfFkTeixg8rKOHIwNXBQ/ITnQxmi5s2D\nzMljNu678GZ4Z/xKqHnsKevjD/o2D8nMFLsSqAI+ecfmy4HDlsT0DGaIGkx6I3X8mms3qX0L\nsQuDyvjl6KGMvw2a/krXOkQUHpB7afy+4ymjm+JTrP7ubY2dtO6mgyci7cO5tjM7dueEvzn7\n1S6+RNQ4Ov3cP/cczBjwWoinysJ3MVQS/6vZEdg2PLSsX9k/IIB1FBQUi1oUVIn9+pr4Y8Fj\nY571FbsSqJKMi5fyQrt3D3b/M6q7xK5aPhp5oKYpMptJqy0LBhqNhhwOh6gVwf+odlTs/M/j\n4+OGhlacm5KYaA+LiHDvVpmGLVto7/18Jc9zZSHYQSUxz4xfseLNNu4pV+6hwz+76jRq6C1q\nUVAFtqurF50Ijh7XFedk1VCOjDtZcgMlf/XB2FeGv/Bq9Hvxu5ILxS4KKq3JM+10SXvWJWSX\ncLZ7J9ftSdV37IDzXWsEpV9QaGhoaH1/VYWZlpzsQpnB4F82zQT41yKj0fg7zxcIDsVClXH5\nP29bHL/xqr73jGEY3ramsSauXHSyYcySLj6ELFBDFRWZiU/ZtLHti69/MNLbfH3XypUz5ykW\nz+qLAW5rFO8ub751evLc2dEHGIbnee/274xspxa7KKgyq9VKygDlw4sl1Bo1Wa0ePPkVwQ6q\nxHJr3/L41Udy/LuOmh09qCn662oY66VVi840il7aBb11NRjv4onT9Zz43vAWciJq1phJe332\n4dO5fYcg2dUgjuS1MxZe9Bsw6e0eoV5Facc2rFg8bZVh0ejmGGqgZvLy8iJ7ScnDGQ67g3Te\nHrxDE4IdVN6DM4umLzjJtX7po38NbVsbf0M10L0rl38tzP981LDPiYhcTuKuzhh2IHzMyo+f\nx/l2NYa3rw9LFBJc9i+oDgkxUKqxgAjBrubgE/fvvlNv6JKxzwUTEYU2MhQkxWw9cmN0czFv\nSgVVp65VS8vdzTMS+ZXOyMvLYwyBtT1XAb6UobLMx5bFn1D2nzV7VDPtf18bqqXgYbO+7O10\nT5ScWTZ5h//bc1582k8valVQOYomTcOYbWm3HdRaQURky0i/LwsOrid2XVApHOcil8tVPi1j\nZaRSoruu5moe0VZ1/PJl8+Ce3kTEp17+ubhRn7YePK6FYAeVVHT2yHlni1ef4dOTkspnetVt\nEuInE7EqqBylT52g8qOwNj8VsV61g4I8+JMSHofAqOHdvp+3dEG9MUNb+Jqvbl95xqff7G4e\nPOQDfx3ToltXv+M747+s9UpkqL4kM2HztpshfaMbiV0XVJm6Xf/n/Katnr9J+1I488veFTuL\nnp3yXIAHC0Cwg0rKTM9w2Y1rp1+qODM8ev1n/dHbA+BZ3p0mfRa7ZvnGL6avLdHWC4+c8tEb\nTXHLghpG02783A/rbtz6w9KPf7Wq/INbDJ45algjfDfXYMrmo+L+SV9u+GLmVpdPg45j42K6\nePTrkeF53pPbAwAAAACBYBw7AAAAAIlAsAMAAACQCAQ7AAAAAIlAsAMAAACQCAQ7AAAAAIlA\nsAMAAACQCAQ7AAAAAIlAsAMAEJXt0tqPPoo/cFfsOgBAChDsAABEZbu0Ni4OwQ4AHgsEOwAA\nAACJwP3oAAAexZtSDu85dDmjSFW3Rff+vVrWdt9/NefIF1+dCBgy7eWwjIPf77uS5QwI7z20\nX7hfxR/IJdkX9u/7KTmnRPtUyx79e4cHyCq+sjXz1O69CbdMqsDQtlH9u4RoHtmuLfPkjl1n\n04uU9Vr1HdK7sZ4RuqEAIEE8AACUcd0/8G5Hf5aIUXlp5QyRtslLX1+z8TzP85enhxFFvfNp\nrzosyVUqGREx/s/OPFXgfq71+jcvhGmIGIWXTi0jIl2L0RtvOdxLLVe/GhKiIiJWLmOISFG3\n94KLVp7njd9EEQW/8v6oxioilmWJiBQhI7bliNF+AKjhcCgWAKBc2uLhQ+cnNYzdkVxYXFRc\nlHVyQaRxc8zgD045ytY4HB8viz2eY7HZiu8d/TSKfoobFrvPQkTWn6YOjP7O3O3fp7KLiswW\n0+3d77dMXzVyUNwFjogsx6cMHLfD3GnWkcxiu70o/eDUiMID741bdtv9snc2fnGp29cX7hfb\ni3NOfNJVnbHhwy+vivIWAEDNJnayBACoLpxH3wokSzqKNAAAA7tJREFUpuuizArz7i/pLiP/\nsUdc7h47eY+lFbrSshZ1lZFi0MZiPn/1ACUpnvsqq8Jzf5nVmsjnjf3O0qVs5OK75ctcCVPD\nVOr+K/JLe+zo6Q+vly9zbH9RRTRorV24pgKAROEcOwAAt6SjR3PJK/hs/JQpD2dmFWoo/8aN\nX6kHERG16t3b8HBh3V69mtPJa9dSyPfECTt1GT68boXXaxgVFTLj35cv3yb+5Ek7tRkw4Kny\nZUzHuTdtc4mICoiI/Lo8+3T5MnlAgA9RYWEhkb8Q7QQA6UKwAwBwy83NJSJj+pUr9yvM9esQ\nFdW4tss9ZTAYKj5Fr9cT5dlsJbm5haR+6qlHg5hOpyOyWCz23FwTUb169f5w03q9vsIUwzBE\nPM//teYAwBMIwQ4AwE2n0xHJB8479HmX31ucQ0RksViIvMvnZWVlEQUEBCi9vORky88vJvIq\nX8jfvXuPKNxgUGg0MiKTyURUu2whZ87JzOf9gip28QEA/EW4eAIAwK1F+/Zqyjl4sOJVC/eW\n9vH3DZ92sWz6wrFjReUL+eu79twmv/btw5iIZyKITu3da3r4VO6nXXsLqElkpKF06aVjx8zl\nC6173mraoPGEvXZBWwQATxoEOwAAN+3gt2PC2OsLR0/dfcvMkdOUvG3y0A8O2CPeGhNRtk7R\ntvfGrLhidBFXdHP7OyMXXqemMdHdWWowctJAX/PW2NeWnLlv44msGbunvLMy26vPlJhwotCR\nE/v7FG1/P2ZDcpGLyHrru5gpG02+Q1/trxWzwQAgOTgUCwBQRtl5zq5v7g18a/7AhgtUaoXd\nZud1bSZ8u35caPkqrYf3TJ3cJmCCVum02Dhl8ODF22Y+IyOiwFdWbb064IV5EzvVmezlo7YX\nFjn1EW9vXvOPICIiw6srN13s+2L8q802Rft62QvMLv92sRsWDdW7L54AAHgsGJyeCwDwiJKs\ns7t3n0zNZ33rN4/s99zTtdx3j7gyo2GbT4MWZx8Yfmfn9yfSrPoG7fsMejZEXeGproLkQ7sO\nXck0s77BrXv269n0kdtSOHOv7PnhyI18NiAkvFu/KPe9JWyX1s754UHnMe/0Dipb8c7eBavO\nBfSb+kZ7dOgBQKUg2AEA/G/Kgt2xCXXELgUA4PfhHDsAAAAAiUCwAwAAAJAIXDwBAPC/qdNz\n0ky5vr1O7DoAAP4QzrEDAAAAkAgcigUAAACQCAQ7AAAAAIlAsAMAAACQCAQ7AAAAAIlAsAMA\nAACQCAQ7AAAAAIlAsAMAAACQiP8DYjkmLcsxA98AAAAASUVORK5CYII=", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "nn <- keras_model_sequential()\n", "nn %>%\n", " layer_dense(units = 200, activation = 'relu', input_shape = 784) %>%\n", " layer_dense(units = 10, activation = 'softmax') # with the softmax the outputs of the neurons indicate the probabilities of the digit classes (see slides)\n", "nn %>% compile(\n", " loss = 'categorical_crossentropy', # this is equivalent to choosing the negative log-likelihood loss (see slides)\n", " optimizer = 'adam', # this is a fancy version of stochastic gradient descent (usually you don't need to tune the learning rate with ADAM)\n", " metrics = c('accuracy') # we also track the accuracy during learning.\n", ")\n", "history <- nn %>% fit(\n", " X.p, to_categorical(data.train$Y), # the target is encoded as a categorical variable (this needs to be done explicitly for keras)\n", " epochs = 10, batch_size = 128,\n", " validation_split = 0\n", ")\n", "plot(history)" ] }, { "cell_type": "markdown", "metadata": { "lines_to_next_cell": 0 }, "source": [ "In the plot you see how the training loss decreases (this is the negative\n", "log-likelihood loss which we optimize) and at the same time the classification\n", "accuracy on the training set increases.\n", "\n", "Next we compute test loss and accuracy:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "lines_to_next_cell": 0 }, "outputs": [ { "data": { "text/html": [ "