Working on your own projects is one of the most exciting things that you can do during your spare time. But, often, it is difficult to know how to actually tackle the task, especially when you begin your activities as an amateur scientist. The experience can then become frustrating and end up with a project that stalls for months.
In this post, I would like to share my experience on the methodology of amateur projects management such that you can increase your chances of success. It is, however, not a miracle recipe and only accounts for my view on the topic at this particular moment. Chances are that it will evolve over time and that if you come back in a few years, I may tell a different story. Still, I believe that the currently presented information are meaningful and could be of some help if you decide to set up your own projects too.
In the next sections, I will first briefly present the two extreme project management models used in the industry before moving on to how I manage my amateur projects. I will assume that the projects are done for the sole maker of the project and that you will work alone although I will also briefly discuss team working as well.
In the industry, you will meet two very different lifecycle models that form two extremes way of thinking. They are the waterfall model and the agile/iterative model. Remember however that these are the pure form of two extremes and that in practice a mix between the two is often used.
The waterfall model is often presented as the old-school, inflexible, approach where everything is planned from the beginning and all the activities chain like a cascade. The waterfall model usually starts with settling the requirements, then goes to the design process followed by the implementation of the design and the tests. There is no going back because the model assumes that every task must be completely fulfilled before the next can begin. A good example of such is the building of a house: you start by identifying the construction site and the major specifications of the house (number of rooms, ft²…) then an architect will make the drawings and the house will finally be built before you occupy it.
The total inverse scenario is the agile/iterative model which is regarded as much more flexible and has become quite popular since a few decades in the computer software industry. In the agile model, things are more uncertain and you advance by small steps known as iterations. Usually, you begin by identifying the major threat to the project and you perform tasks to mitigate the associated risks. Example of threats in an engineering project could be “is the motherboard of computer X fast enough to cope with the throughput of the high-speed camera Y?”. It is a threat to the project because if it fails, the whole project will ultimately fail. In this example, there would be different ways to mitigate the risk: one could be to contact the manufacturer of the motherboard and of the camera to get advices, or, more ideally, to perform the test before buying the camera and the computer. Once you have solved the most important threat to the project, you identify the next one and mitigate it as well. There are a lot of different ways to mitigate a risk which range from checking the literature to performing simulations or building prototypes. This risk identification-mitigation process continues iteratively until you reach a solution where your application has been successfully implemented.
Obviously, each model has its own application domain. Waterfall is more suited when there is little unknown and when you have sufficient experience in the realisation to be done. Agile/iterative is best suited when you are sailing in dark waters and where no previous knowledge exists. It is important to understand the limit of applicability of each model because it would be silly to stick with a model that is clearly not suited to the problem faced. For instance, it would be unwise to use an agile method to build a house because there is no need to go through three or four prototypes since the construction of houses is a well-known process. At the opposite, you probably do not want to apply the waterfall model to a PhD thesis because you simply cannot plan research activities on a period of several years (you do not schedule “month 36: big breakthrough; month 37: patent filling; month 39: mass production begins; …”). Strangely enough, however, I had to submit such months-to-months schedule twice when applying for grants. This is because the people who actually pay for the application wants to know when they will get results and this is something that the waterfall model offers by essence while the agile/iterative model does not. You need to be aware of it but the good news is that amateur projects usually do not have schedule to comply with and we can therefore tailor a method that is really suited to our needs without having to worry about filling advancement reports.
Finally, and because amateur projects have mixed amounts of knowns and unknowns (they are not complete reproduction of existing materials –in which case you wouldn’t probably be doing it– but they are not 100% innovative either), a compromise between the rigid structure of the waterfall and the iterative model can be used. It is presented in the next section.
Here is my approach on how to manage amateur projects. It does not account for team working and consider that the person developing the application will be its user. This has important consequences because one of the major difficulties in project management is to understand what are the actual needs of the client who, by the way, has to be challenged on its initial requirements if you want to be sure that he is actually asking for what he needs. When the user is also the maker, things are more straightforward because the requirements and specifications are better understood and never over or under-engineered.
I decided not to include team working in this article because it makes things more complicated. Actually, I would recommend to work solo during the first years until you master the project development methodology before moving to team working. Only then you may consider to work as a team and I would then recommend to select easy projects in a first place. This is because team working is a complex thing on its own and you do not want to mix two source of difficulties (complex project plus complex management). When working in teams, everybody does not always share the same idea of the project and everybody is not always up to date with the latest modifications. Keeping the coherence then requires frequent meeting sessions and keeping track of foreseen/committed actions in lists so that there is no misunderstanding on who should have done what and how. Also, remember that the management complexity will increase as the square of the number of people involved in the project because the number of individual communication channels increases by that much (in a group of two people there is only one possible communication channel, but with 6 people you already have 35 different possible interactions). Finally, and at the opposite of what happens in the industry, you cannot force people to work on your project. People will do it for free and you usually have little authority on them (this is especially true with group of students where people involves themselves at very different levels). I had the occasion to work on projects with up to 30 people and we made less progress in 6 months that what we did in 2 months with the 3 most motivated people afterwards. With so large teams, you will spend all your time communicating with people and trying to motivate them such that you have no more time to spend on the technical aspects of the project. So, my recommendations on that would be to work with a very small team of friends, typically 3 people in total, and to select easy projects first.
The approach presented here is based on an iterative application of a waterfall-like model which starts with specifications identification then goes to design, procurement, integrations and tests. From the results of the tests, the earlier steps are updated until the iteration converge to a satisfactory outcome. Each iteration will produce outcomes that can be fed into the next iteration until you reach a satisfactory stage. In my experience, one to three iterations are often necessary before reaching something suitable for publication, depending on the complexity of the project.
The very first thing to do once you have identified what you would like to do for a project is to set specifications on paper and make a rough drawing with the major components that could make your project work. It is also a good idea at this stage to have a look at the literature (e.g. google image, engineering books…) to see how other people have implemented it before.
Specifications are all the inputs that will apply to your project. They are made of either requirement (what the user wants), design choices (either arbitrary or on purpose to cover a requirement) and environmental factors (things that you do not have control on). Every time you do something on the project, it should be backed-up by a specification and you have to keep track of them, review them frequently and, most importantly, to challenge them. There is nothing worse than a poorly chosen or outdated specification because it will make your design more complex than it ought to be.
To understand the necessity of listing specifications, you have to see your project as a very large space of potential solutions. This solution space is unfortunately way too large for our brain to deal with it and it is not possible to evaluate all the possible solutions until you find one that is satisfactory. Every time you put a new specification on the project you trim big parts of this solution space and leave something that is smaller and already more prone to evaluation. Ideally, you will have to put specifications until there is only one solution left in the design space and that particular solution will be your final design. Unfortunately, there is no way to actually evaluate the size or content of the design space in practice so we will just keep adding specifications until there is no more choices left in the solutions that we can think of.
Note that at this stage we are only talking about concepts and not detailed design but the big picture is there. We already know that to achieve the goal with such and such optical elements or gearings arranged in that particular fashion and we leave the actual design of each individual elements for later (e.g. exact radius of curvature for the lenses, type of coating to use, number of teeth for the gearing, type of operational amplifiers…).
But every time we change a specification the rough design may change as well. Changing the maximum power of a LED in a circuit from 20 mA to 500 mA will have dramatic consequences on the design because the kind of technology used for both of these ranges are completely different in terms of power management. This is why it is important to challenge every specification to be sure they are correct and do not take the project to a wrong road. Keeping track of specifications in a document and reviewing it frequently is the best way to keep the project on track.
When you have completed your rough design sketch you will end up with a lot of interacting parts for which you have to decide if you are going to purchase the element off-the-shelve or if you are going to make a custom development for them. For instance, in a project you may have a laser diode driven at constant current and collimated with a lens. Usually, you will buy the laser diode off-the-shelve (unless your project is about building a laser diode, but this is unlikely at this stage) but you may decide if you will make the laser driver circuit yourself or if you will buy one. Same for the lens: you can choose to buy one from stock components (most likely hypothesis), make an optimized lens and order a custom lens, or completely make the lens yourself from, say, PMMA plastic on a lathe. Choosing between build vs. buy is something that is up to you and depend on your motivations. In my experience, building takes more time, is more expensive and offer less performances than buying a professional solution but when you build something you get a better understanding on how it actually works.
My advice for amateur projects is to break the project into sub-projects until there are only off-the-shelve components remaining. Each sub-project is then processed individually, one at a time, until satisfactory. This means that you have to completely focus and write rough design, specifications, detailed design, procurement, tests, reports… for each custom component that is involved in your main project. In my example, I would put the main project (laser diode + driver + lens + …) on hold and dig the laser driver circuit first, write detailed specifications about it and focus only on this driver until I have built one that is working satisfactorily. Only after that, I will resume the main project.
Because building is a good way of understanding things but usually offers less performances than off-the-shelve components, what I sometimes do is to create a custom part to understand how it works but, then, I will order one off-the-shelve to be used in other project just because it will be more reliable or have much better overall specifications in terms of noise, power...
To conclude, a good way to tackle amateur projects is to develop a project as a tree where each custom part becomes a node and the leaves are off-the-shelve components. Then, you select the deepest node of the tree and create a subproject for it, putting all the rest in pause. When you have completed the sub-project, you resume the main project and select the next node of the tree in an iterative way. So, our waterfall-like model actually becomes a cascade of iteration processes where each problem is treated one at a time until we converge to the final application. This is also a good way to keep you motivated because you can see the improvements everytime you close a sub-project while you would have had the impression to stall if you had kept the project as a whole.
Once you have identified the building blocks with their general properties, it is time to details things a little bit. Although you are going to make a lot of design choices here, it is not necessary to link them to explicit specifications because that would be overkill. For instance, when making a printed circuit board, you route tracks in such manner as to prevent noise, power losses, cross-talk, reactances… but you will not create explicit specifications for that because you would end up with a way too exhaustive list that will be difficult to read after. The same is true for optimization of optical components, coating selection and so on. They all obey to general principle of optimization that you will follow from projects to projects. The only exception to that is if some point is very specific to the project. For instance, you may create a specification about noise level (e.g. “output noise voltage should be lower than 100 µVolts at 3 MHz bandwidth) in a project about a low noise amplifier for a rocket bench test.
Still, it is important that even if you do not create explicit specifications about these detailed design choices that you should be able to motivate them if requested. Nothing should be made without consideration as every choice has an impact on the project performances.
At the end of the detailed design you should have a frozen version of all the technical aspects of your project. This typically includes mechanical, electrical, optical… drawings with all the figures put on them. Next stage will be the procurement and the integration of the parts so it is important that you are 100% sure that everything will work as expected.
To prevent any issues my recommendation is to double or triple check everything. Check the datasheets for maximum ratings, input voltage, output swings, bandwidth, tolerances… When possible, simulate using either proprietary softwares (such as PSPICE for electronics) or from custom simulation models built from Matlab, Simulink, Excel… If you have components on hand, build a breadboard also. On that side, remember that simulations are only computer models that have limits in their application domain and it happened to me several times that a circuit simulation went just fine in PSPICE but completely screwed up when implemented in the real world!
This is probably a point where we will fall back a bit more into the iterative model because, to mitigate design risk, it will be necessary to order a few components for breadboarding at some occasions. This is just fine and remember it is best to lose a bit of time and money at this stage rather than having a dramatic failure and order for 200 EUR of unusable PCBs because of a design flaw. When you are done with the test, just put the components back into your global stock so that you will be able to perform tests for other projects later without having to purchase anything. Do not misunderstand me however: you should only buy parts that have a direct usability for the project and do not stockpile stuff just “in case of”. I have plenty of ICs, gears and motors here that are still waiting in their original plastic bag for years because someday I was stupid enough to think that these would surely be helpful someday. To keep things well separated, a good trick is to store all parts of a given project in a spate plastic box with a label on it. That way, you will never take in the stock of a project to build another one and get stuck when resuming the initial project because you are out of components.
When your designs are done, I recommend to let them cool for a few days and, when you have a fresh mind, to review them one more time to find potential flaws. If you find something, just repeat the process a few days after and so on until you are satisfactory with the design and ready for the procurement.
Once you have your detailed designs, you are ready for procurement. At this stage, do yourself a favour and go to professional companies and do not go to DIY stores or the like. In my experience, when going to professional companies you get much better quality for the same price and you also have much more choice. For instance, when working in fluidics I first used a lot of silicon tubing bought from pet stores (aquarium tubing). They had a lot of different accessories such as valves, “Ts”… and I was happy with them until I discovered Cole-Parmer which had the same items (and more) in much better quality and at the same price. There are a lot of professional stuff out there that is just waiting for you to pick them up. The only limitation is that some companies will not accept to deal with you unless you are a company yourself. Fortunately, they have competitors and some of them will accept your orders. Just pick your phone and call them explaining your situation; most of the time they will try to help you. If necessary, make a blog and buy a dot com address so that you present better.
Ordering off-the-shelve components is done primarily based on their datasheets so it is rather straightforward. Custom orders, such as PCBs, are usually done from drawings (e.g. Gerber/Excellion files) or, if you are more advanced, from a detailed document called the procurement specifications where you state exactly what you want. Think of this as you initiating a project at the supplier where the requirement inputs will be all the things you listed in the procurement specification document. Also, when order customs it is important to determine before the purchase if you are going to check the quality of the product or not or if the company is going to do it. For instance, most PCB manufacturers will make an electrical check on the tracks and performs a visual inspection for the overall quality. It is up to you to decide to double check this or to perform other checks but it is important that you do them at the moment you receive the parts such that if anything is not compliant with the purchase you can get in touch with the supplier as fast as possible.
When you send something for production, it is also important that you freeze the design and assign it a revision number. Then, if you add modifications to the design, you create a second revision number and so on. The easiest way is to always keep a version known as the working copy where you do all the changes and when you have to freeze the design you simply copy the working copy folder and rename it with a revision number (e.g. lightdriver-rev1, lightdriver-rev2…). Revisions should never be changed and, unless clearly stated in the project, the latest revision is always the one to select for procurement. This is true for revisions but it is also true for all the documents that you produce, including the specifications. When you update specifications, you have to keep track of what change have been done at what time and for which reason. You do not need fancy softwares to do that, Windows folders are enough for the CAD files and a word processing or spreadsheet document works well for the specifications. Finally, a good trick to enforce that revisions are not changed is to put the revision folders in read only as you create them.
One last thing that I really do recommend is to keep track of your orders with a least the following information: date of purchase, name of the supplier, description of the product, date of reception, date of payment. This will allow you to keep track of pending orders and pending invoices. At first you will feel that his is overkill but trust me, it is mandatory when you place several orders a month and do not have 100% of your mind focused on that because you have to take care of other things as well. Also, you will quickly discover the joy of back-orders where you get only half of your order and see that the other half will be shipped in two weeks because it is not in stock. It is also a good idea to group incoming paperwork because you are going to receive a lot of invoices, delivery notes and order confirmation. For instance, with a simple order at Thorlabs I usually end up with 10 pages of data: order acknowledgement, End User Statement documents that I have to sign, delivery note and invoice. However, I do not necessarily recommend to group them by project as one could have expected. This is based on the experience that, in amateur projects, a lot of orders that you place do often comprise components for several projects at the same time. With such grouped orders, it is difficult to decide where to put the delivery notes/invoices and you do not want to make hundreds of copies of them just for the fun of classifying the data.
Once you finally received and inspected all the incoming parts, you can start integrating your project. If you have several interacting parts, proceed one step at a time and check that everything is working fine. In electronics, a good oscilloscope and signal generator is often of great help but it is not mandatory (although very, very, nice to have). Also, start at reduced power and gradually ramp to see that everything is okay and that no part gets too hot, melt or even, catch fire (I had some nice puff of smoke in the past on a couple of projects). In that concern, a ringing power supply is usually the sign that you have a shortcut somewhere and that you better have to unpower the circuit very quickly. The same is true in other fields such as optics where it is a good idea to start your light source at reduced power in a first place before turning it on to full power to keep your eyes protected. If you have time, I strongly recommend you to take pictures and store them on your hard drive where you put the files of the project you are working on. It helps a lot when you have to rebuild a new model 6 months later or simply to illustrate your reports.
Concerning the tests, you should ideally check all the listed specifications. So, if you have a spec on noise level, you will have to test it. Same if you have a spec on optical resolution, you will have to measure it as well. Remember that the result of a test is either positive or negative. If, for some reasons, you cannot perform a test (e.g. you do not have the hardware to do it), you will have to decide on whatever it is an important outcome for the project or not and document it.
Most important of all, any failure to perform as initially foreseen should be reported as a non-conformance and documented. When documenting a non-conformance, you should first state clearly what is the problem (e.g. the circuit output oscillates on high gain settings), what is the most probable cause (e.g. under-damped amplificatory) and the severity of the non-conformance (minor/major/critical). Minor non-conformances are those that can be solved by a hot-fix on the ordered parts (e.g. solder a capacitor on top of the high gain resistor). Major non-conformances are those that requires stopping the integration and order new parts or replacement parts. Critical non-conformances are those that just kill the whole project because you suddenly realize that you completely misunderstood a fundamental concept and that it will never work that way. For the two former cases, you also have to document how to resolve the problem via a hot-fix if possible (minor non-conformance) and, more generally, how to fix the issue in the original design such that next time you order the parts the problem will be solved. My advice here is to fill-in a new form called a change request which state what change should be made and for which reason. The good thing with change requests is that you can raise one whenever you feel that an improvement is possible on a project. So you do fill-in a change request when running into a non-conformance, but you can also fill-in a change request even if everything went fine but saw some room for improvements (e.g. use a 10-turns trimmer for better accuracy instead of a single trimmer in the offset adjustment).
Once you have performed all the tests, you will have to review all the pending change requests and decide if you implement them or if you are happy with the current prototype and decide to move on to the reporting. Bear in mind that these iterative cycles are usually never ending and that even after the 10th iteration you will still find room for improvement. So it is important to decide when you have enough information to go to reporting. Note that you are still free to re-open a project after a few months/years, implement some changes and make a second reporting about the changes and so on.
When you are happy with the project outcome, it is time to report everything you have done. Your report should include all the information necessary to reproduce the project. You may do the report for yourself or for other people as well (as I am doing here on this blog).
Long ago, when I started experimenting, I did not kept track of what I made because I thought I could remember exactly how I performed the experiments. The truth is that, after a few months or years, I had completely forgotten the info and was unable to re-run the experiment. Sometimes the experiment is quick and just consist of reproducing something that is already existing on the Internet and you feel like you do not need to keep track of that because the day you need to re-run the experiment you could just go to Internet again. Unfortunately, websites tend to disappear and the information get lost as well. So, it is important to copy everything that is necessary and write something about what you have done.
Reporting should also be the occasion to draw conclusions on what are the actual (measured) performance vs. the expected ones and how much money and time has been spent on the project. I think these two last factors are important to monitor because our resources as amateur are quite limited (in both time and money) and it is a good thing to have a check at them before going crazy and spend more than you earn in your hobbies.
To summarize the methodology, you should always respect the following rules:
- Write down your specifications and challenge them;
- Break your project into a tree of sub-projects and tackle one difficulty at a time;
- Double-check your designs, simulate them when possible or build breadboards to assess their performances;
- Freeze your designs each time you place an order by giving them a revision number;
- Test each specification you have set, report bugs and potential improvements;
- Iterate your design until you reach a satisfactory state;
- Report your work and draw conclusions.
Following these rules will increase your chances of success by offering a strong methodological framework. It is how I have been working since the last few years and I do sincerely trust that it is a good way to tackle most of the amateur projects as well. So, feel free to contact me to share your thoughts about it!