Other JISC projects - open source, sustainability ... the lot

Following my post last week, related to whether it is feasible for WebPA to become sustainable and the impact of the funding, a post popped up on Rowins blog on essentially the same topic.

Rowin's post is about three JISC funded assessment tools. The three tools though have left becoming open source till their funding has ceased. So what is the future for these projects, well I don't know. I fear that, although there are seven developers and a sourceforge area, there is no wider community to support the future of these projects. Further funding would possibly extend their lives, but as pointed out time and time again, with no community to support the projects, is there a future in the short term worth paying out for.

I would love to know from all of the JISC funded projects, funded at the same time as WebPA;

• how many have released their code under open source,
• how many are beginning to build up a community to support their tools,
• who has waited till the end of the funding to release their tools,
• has any one really shouted loud enough to gather interest in the tools, to build the community's,
• is there any tool which is likely to reach sustainability.
  

I know that WebPA is the exception to all of those, but at times it has been a fight to get every ones buy in to the open source ethos and way of doing things. Some of the barriers and limitations on to the project have been discussed in this blog. I hope to continue raising these sometimes awkward points, at least till the funding ends for us too, or in the perfect world we reach sustainability.

One innocent little action...

In the run up to christmas I suggested that it would be good for the project to enter the IMS Global Learning Impact Awards. In doing this I landed my self a huge chunk of work in filling out the application. The application was not as straight forward as I had hoped and it even ate into my christmas holiday. But the application went in, with some in put form the WebPA evengelists here at Loughborough and the JISC Cetis Assessment co-ordinator.

The new year came and went. Then at the beginning of Feburary news came in that we had been accepted as a finalist. There were a couple of weeks to decide whether to progress to the finals or not. Now progression has been agreed and in the space of a few days news is already getting around.

Firstly, the Cetis co-ordinator sent an email with congratulations and now JISC are asking for a quote for a press release on how the project feels. Hopefully all this publicity will work for the project and draw in more institutions that will want to use the tool. All in all, not bad for a little open source project, bearly a year old!

Feeding user needs into WebPA

Over the years that WebPA has developed, at Loughborough University, the users have always requested new features to be added. This developed into a long list of academic wishes, but it has so far not been evaluated to show if the whole user community requires the features listed. Now, seems an appropriate time as the project are looking to update the road map.

The road map document lets the projects community members see where the project is intending to go for the next phase of work. WebPA tries to update the road map every six months of the projects current funding period. I hope that in this phase the community needs for WebPA will be more clearly reflected in the development to take place.

In order to ensure that the community needs are reflected the current wish list was sent to all active users at Loughborough University. The community where sent a list of requirements that had previously be gathered and asked to select their top three. If their most important requirements where not on the list the community was asked to feed back what they needed. From all the emails sent there has been a 23% response rate. While this is not wonderful it is sufficient, well we aren't going to statistically analyse the information. From this I have identified a list of requirements to take the development of WebPA forward while ensuring the users needs are met. These requirements may not be pedagogically sound in most cases, but as an open source project I think that we can be forgiven this if we are meeting user need.

I do think that it will be interesting to re do this task as the project community grows. But in that growth I feel that more work will be needed to ensure that the community of users are aware of the WebPA sourceforge area, and how features can be requested.

WebPA intergration with VLEs

I have in the past twenty four hours been asked about the projects plans to develop integration modules for a particular VLE. I knew that this question would arrive eventually, but I still found my self relatively under prepared to answer it.

What I should be doing is asking the person who sent the query to then develop the solution. That is how open source software works, right. But this is also a project with deliverables which in the perfect world have to be met. I also don't want to scare our potential pilot institutions away from the project. As a result I did reply with a concise statement "that this may be considered in the future". But there are a lot more VLEs than one, so I do desperately need the contribution to the project.

At the beginning of the week I read an article about how to run a successful open source project by approaching the development modularly. This is said to encourage users to write their own plug ins. This method has been a success for other open source project such as Mozilla FireFox, Drupal etc. But from a little investigation this is also a successful model for established software companies and one in particular Blackboard, where the community contribute "building blocks". As a result of this, I really need to be encouraging the current developing user community into becoming a contribution community, creating these needed plug ins/modules/widgets.

I have a feeling this maybe more successful than I thought it would. With the Web 2.0 movement, especially in Education, more and more individuals are developing these little code elements to accomplish tasks they need doing. Therefore, in an ideal world, those individuals should be happy enough to add modules for WebPA. This then has a knock on effect for me... I need to now go and assess the code to see how easy it will be for individuals to contribute. I will also need to make the changes to the system in order for this to work. Even if we do not see the benefits now, I hope that we see them some where in the future.

NOTE: I initiated a conversation on the Community-Development JISCmail list. The full conversation to date can be found at http://jiscmail.ac.uk/lists/COMMUNITY-DEVELOPMENT.html

Measuring the success of WebPA

Last week I attended a JISC workshop on evaluating JISC funded projects. Although a lot of the evaluation seemed to be steered towards the student experience, it did raise a number of points related to evaluating the project. These are especially valid for those projects where a large proportion is related to the development of software.

In the past I have sat in no end of meetings where evaluation has been discussed, and the software is quite often overlooked, or in many cases the evaluation of software is not understood. I have seen no end of project submissions where the evaluation for the software produced is out lined in terms of "the code will be peer reviewed" but as an opensource project that will happen over time and will be carried out by the community that develops for the project rather that a nominated project or organisation. Anyway getting back to the software the evaluation should be in relation to how widely the system is now used and the attitudes of those using the system.

One phrase that came out of the workshop which has stuck with me is "Metrics for Managers". I think this is a good place to start to highlight the types of information that can show the success of WebPA (after all it will be a success). For those who are interested in what WebPA can do for the institution we need to look at the student retention rates, rather that the rather softer student experience. By evaluation the retention rates we may be able to show that with the use of WebPA for peer assessment, the retention rates increase. However, this may be far in advance of what can be done in the time remaining for the delivery of this project.

There are other measures that we can use for the evaluation of WebPA, link the ones that I have highlighted in the past with out even thinking about evaluation. We are already able to collect data related to the download rate of WebPA, we just need to be able to marry this back in to the number of Institutions across the UK and then by sector that have adopted WebPA. This is more difficult that it may seem to begin with. For a start yes we can find out the number of downloads for the software we are receiving (12 to date), but due to the privacy policy for Sourceforge I can not find out any information about who is downloading. This then leads on to the second element, how will we know who is adopting WebPA either on an institutional level or individually in their teaching. The only what of measuring this will be the metrics that we can gather from other areas of the sourceforge system and form contact that we have with institutions and academics. But I do think that these metrics will be in some cases difficult to obtain and also miss out some vital information, however, they will be useful anyway.

Even though evaluation is normally left till the end of the project I think the JISC timing for this event was appropriate. I think that it will trigger project teams to start thinking about what they can evaluate and how, in turn this will enable the teams to start collecting data sooner, rather than scraping around for information and trying to make it fit towards the conclusion of the project funding.

Are we a community project?

I have been listening to the latest 'Open Season' on 'the Register' and they are discussing Yahoos purchase of Zimbra, but they keep coming back to the marketing and the type of open source projects that are around. One of the terms that I keep picking up on is 'community project'. At the moment I am not too sure what a community project is. I have done a quick search on google and there is no defined definition. But what exactly is a community project?

Well I understand a community project to be a project that benefits a particular group of people by fulfilling a specific task that is common to all. From this I see a CoP developing around the project and I think this is the way that I hope WebPA is moving.

The other concept that I picked up on that may help define a 'community project' is the user base. From my understanding a community project within the open source world seems to measured by the number of users that the project has. For me this maybe a negative view of projects. For example WebPA will never have a user base say the size of Moodle, but that should not mean that we are not a successful community project.

As the open source movement continues to develop at the rapid speed that it does I think the original value of open source may be lost.