Homec4science
Diffusion libphutil (stable) Browse

libphutil/src/docs/drafts/test_plan.txt
bc1c139c818bstable

Last Change
Raw File

test_plan.txt
View Options

This document describes things you should think about when developing a test
plan.
= Overview =
When you send a revision for review in Differential you must include a test
plan. A test plan is a repeatable list of steps which document what you have
done to verify the behavior of a change. A good test plan convinces a reviewer
that you have been thorough in making sure your change works as intended and
has enough detail to allow someone unfamiliar with your change to verify its
behavior.
= All Changes =
* <b>Error Handling:</b> Are errors detected and handled properly? How does your change deal with error cases? Did you test them and make sure you got the right error messages and the right behavior? It's important that you test what happens when things go wrong, not just that the change works if everything else also works.
* <b>Service Impact:</b> How does your change affect services like memcache, thrift, and the databases? Are you adding new cachekeys or queries? Will this change add a lot of load to services? Use Cacheobserver to understand the service impact of your change.
* <b>Performance:</b> How does your change affect performance? Use tools like XHProf and Microbench to measure the effects of your change. **NOTE**: If your change is a performance-motivated change, you **MUST** include profiles in your test plan proving that you have improved performance.
* <b>Gating:</b> Is your change gated (e.g., by gatekeeper or sitevars) so you can revert it quickly if necessary? Should it be? Have you tested that the gating works properly?
* <b>Unit Tests:</b> Is your change adequately covered by unit tests? Could you improve test coverage? If you're fixing a bug, did you add a test to prevent it from happening again? Are the unit tests testing just the code in question, or would a failure of a database or network service cause your test to fail?
* <b>Push Robustness:</b> Is your change robust against version issues caused by the push? For instance, if you change a form, users may post from the old version to the new version (or vice versa) during the push. If this occurs, will anything really bad happen?
* <b>Concurrent Change Robustness:</b> If you're making a refactoring change, is it robust against people introducing new calls between the time you started the change and when you commit it? For example, if you change the parameter order of some function from <code>f(a, b)</code> to <code>f(b, a)</code> and a new callsite is introduced in the meantime, could it go unnoticed? How bad would that be? (Because of this risk, you should almost never make parameter order changes.)
* <b>Revert Plan:</b> If your change needs to be reverted and you aren't around, are any special steps or considerations that the reverter needs to know about? If there are, make sure they're adequately described in the "Revert Plan" field so someone without any knowledge of your patch (but with a general knowledge of Facebook systems) can successfully revert your change.
* <b>Security:</b> Is your change robust against XSS, CSRF, and injection attacks? Are you verifying the user has the right capabilities or permissions? Are you consistently treating user data as untrustworthy? Are you escaping data properly, and using dangerous functions (like HTML()) only when they are strictly necessary?
* <b>Architecture:</b> Is this the right change? Could there be a better way to solve the problem? Have you talked to (or added as reviewers) domain experts if you aren't one yourself? What are the limitations of this solution? What tradeoffs did you make, and why?
= Frontend / User-Facing Changes =
* <b>Static Resources:</b> Will your change cause us to serve more JS or CSS? Can you use less JS/CSS, or reuse more? Can you replace JS with primer-based functionality?
* <b>Browsers:</b> Have you tested your change in multiple browsers? You should test IE6, IE7, IE8, Safari, Firefox, Chrome and Opera if you are changing JS or CSS.
c4science · Help