When can you say you have done enough testing

Being the only developer on an open source project at the moment, I have all the roles to perform as part of the software development process. These roles include being the developer, developing the code and carrying out the testing to ensure that the code works as specified. But also being the testing team as well, doing all the formalised testing.

From my days at university and my module on testing I can only remember so much. But which testing should I be doing. From my point of view I can not sufficiently carry out any white box testing, even though I know all the internal code and workings of the system. But as with all things in life time is short, even though there are tools out there such as phpunit, but do I have enough time to work out how it works and then devote the time to implementing the testing routines. I think that it would be very beneficial for the project in the future to have this formalised unit testing, but at the moment it may be just a bit too much for one person.

However, there are other forms of testing, including black box testing. As there is more focus on what the system user sees and experiences, then I think that this is more important. This importance comes from the type of project that we are and also in that we want to build a community around our tool. Although accuracy is important in the algorithm that is used to create the scores for the students, I can fairly, happily say that this has been repeatedly tested and verified in the use of WebPA at Loughborough University.

So now I have worked out the direction in which the testing should go, when can I say I have done enough to ensure I meet the release criteria (which I have defined). How many of the steps of the testing process do I need to fulfill in order to guarantee that the software has been sufficiently tested, are there any steps that I can miss. Oh, and how does this fit in with the development methodology that was chosen for the project, which is agile.

At the moment I have carried out over 30 distinct documented tests, and subsequently corrected the code. Is this enough to move to the rest of the quality assurance process? Do I need to do more? How much time have I left to do any further testing? I have no definite answer for any of my questions, and I seem to face the same questions everything I work on a large project, with out the back up of a testing and QA team. I will just have to rely on time constraints and gut feeling for now.