eNews Volume 7 Number 1 February 2008
Web Projectshow are they different?
The latest ISBSG Special Analysis Report takes a look at web projects and compares them to non-web projects to gain an understanding of any factors that might make them different. We hope that this information will help improve the planning and management of your web projects.
The report includes:
- A comparison of the key project measures
- A comparison of Development Phase Effort
- A Techniques & Tools Usage comparison
- Team Sizes for web project size ranges
The report reveals how web projects differ in their development environment. Almost 50% of web projects use Java as their main
language. .Net, C# and ASP also have significant usage in web projects.
Web projects also make more use of prototyping tools, which fits with the sort of interaction between user and developer that has to take place in these projects.
eNews Volume 6 Number 5 December 2007
Techniques & Toolstheir impact on projectsSpecial Report II
In the first paper on techniques & tools we primarily reported on their impact on the Project Delivery Rate of projects. The research for that report led to more questions about what impact tools & techniques have on other aspects of projects.
In this report we look beyond PDR to provide information about Speed of Delivery; Defect Density; Team Sizes and any changes in project Phase Ratios that have resulted from the use of a technique or tool. With the co-operation of Capers Jones from Software Productivity Research we are also able to compare our results with the results of his research.
This report provides some interesting insights into the positive impact of CMM(I) environments, Prototyping, RAD and JAD while also revealing the generally low usage of almost all techniques and tools.
The report is available free to ISBSG Web Subscribers or can be purchased from www.isbsg.org/products
eNews Volume 6 Number 4 September 2007
Collecting valuable data
Everyone wants information but no one wants to collect data!
Obviously that statement is not strictly true, as the ISBSG has managed to gather data on 4,150 software projects from more than 20 countries. But far more people want to devour information from the ISBSG analyses than provide the data on which the analyses are based. One of the roles of the ISBSG is to provide data collection questionnaires (DCQs). These DCQs are effectively the data collection standards that present questions to gather Software Project and Maintenance & Support data
that can be stored in the ISBSG Repositories and then used for analysis.
The ISBSG DCQs are not restricted to the ISBSG. Any organisation can download a copy of the relevant DCQ from the ISBSG web site and use it as the basis of their own data collection. There is no charge for the down load as the ISBSG wants to encourage as many people as possible to adopt its data collection standards. Naturally the hope is that this initiative will result in an even greater number of projects and applications being sent to the ISBSG.
All of the ISBSG DCQs have been developed by an international working group, thereby ensuring broad acceptance of the data collection standards.
Download a Data Collection Questionnaire to help get your data collection started and while you are there download Practical Uses of the ISBSG Data from:
http://cts.vresp.com/c/?InternationalSoftwar/f6ad011052/TEST/20962338dd
eNews Volume 6 Number 3 June 2007
Are there any silver bullets? A Special Analysis Report on software Tools & Techniques
In recent months we have received a number of questions about the impact of various software development tools and techniques. These questions prompted us to do an analysis of the projects contained in Release 10 of the ISBSG data.
What works and what doesn't? Are there really any silver bullets? In this latest report we look at what tools and techniques are being used and their impact on sofware development productivity. Under the heading of Techniques we include: methodologies, modelling, prototyping, RAD, OO plus the use of CMM(I) and standards. Under Tools we investigate the use of project management tools, CASE tools and requirements/specification tools.
The report is available free for subscribers or can be purchased from:
http://cts.vresp.com/c/?InternationalSoftwar/c6fbefabea/23010cf16b/54278a8f=77
Please note that the ISBSG does not evaluate individual software tools.
Corporate Licenses for Special Reports:
To allow organisations to distribute copies of the ISBSG Special Analysis Reports without restriction we have introduced a corporate license. A single license fee of US$250 allows a special report to be distributed organisation wide.
eNews Volume 6 Number 2 March 2007
Special Analysis Report on Phase Ratios
The ISBSG collects data on the effort spent on each phase of a project. In 2006 Design was added to the five existing phases of: Plan, Specify, Build, Test and Implement. Previously high level design was recorded as part of the Specification phase and detailed design as part of the Build phase. Now that the repository has sufficient projects with effort recorded against all six phases, we have produced a special report that looks at the phase ratio break-down for enhancement and new development projects, compares these with the data from the five phase projects and provides an example of how to use these ratios in project planning.
Treating Design as a separate phase has helped obtain a finer-grained understanding of phase percentages without distorting the numbers measured. Some useful rules of thumb have emerged: Planning takes around 9% of the project total effort as does Specification; Design is around 14% and Implementation 6%. The Build and Test phases show greater variations between Enhancement and New Development projects.
Knowledge of these ratios can be very useful in project estimation & planning, project management and project benchmarking.
The report is available free for subscribers or can be purchased from:
http://r.vresp.com/?InternationalSoftwar/51a09c5031/891624/TEST/TEST
Corporate Licenses for Special Reports:
To allow organisations to distribute copies of the ISBSG Special Analysis Reports without restriction we have introduced a corporate license. A single license fee of US$250 allows a special report to be distributed organisation wide.
eNews Volume 6 Number 1 January 2007
Release 10 of the ISBSG Data is now available
The ISBSG Estimating, Benchmarking and Research Suite Release 10 is now available. This CD has data on 4,006 projects plus the Early Estimate Checker tool. To review the content of the CD you can download a copy of the data demographics & field descriptions from: http://r.vresp.com/?InternationalSoftwar/faaf3f6ae9/829342/TEST/TEST
You can order the CD from: http://r.vresp.com/?InternationalSoftwar/2daf476978/829342/TEST/TEST
If you purchased Release 9 after June 30 2006 you can upgrade to R10 for US$420. To order the upgrade, email: admin@isbsg.org
This powerful suite can be used by:
- Project Managers and anyone involved in software estimation. The Early Estimate Checker tool provides estimates based on regression analysis of the ISBSG data, while the data itself can be used for comparison and analogy based estimation.
- Users of commercial estimation tools. Some commercial estimation tools, like Predictor provide for the use of the ISBSG data contained on the CD.
- I.T. Planners for development analysis. The ISBSG data can be used to analyse and compare the performance of
languages, tools, methods and techniques. This allows decisions to be made about the most appropriate hardware/software platforms for a project or development environment.
- Software Metrics Consultants. The ISBSG data allows consultants to provide their customers with benchmarking services, estimation information and guidance on the factors that impact development productivity.
- Standards Assessment, Process Improvement and Quality Assurance, (CMMI, ISO), Consultants. The ISBSG data provides consultants with a base for their client assessments plus valuable benchmarking data.
- Outsourcing Managers. The data provides a benchmark that can be used as a base for outsourcing contracts and performance agreements.
- Submitters of projects. Organisations that have submitted projects to the ISBSG Repository can benchmark their projects against like projects and can analyse their own sub-set of projects within the ISBSG data.
- Researchers. Academics can make use of the ISBSG data in their research work.
eNews Volume 5 Number 5 October 2006
Using the ISBSG Data to help Project Management
There are many uses that can be made of the ISBSG data beyond Estimating and Benchmarking. Project Managers, particularly, can benefit from some or all of the following:
- Verification of completeness of software requirements
- Early lifecycle estimation of software size, effort and cost
- Early lifecycle estimation check of effort, duration & speed of delivery
- Awareness of project risk, checking the reality of estimates
- Determining an appropriate project team size
- Checking project duration
- Planning and monitoring projects
- Acquiring custom-built software based on a price per functional unit.
- Benchmarking your project's performance, post implementation
We have produced a short presentation that introduces these data uses. You can download this presentation, free, from here
eNews Volume 5 Number 3 July 2006
Planning ProjectsRole Effort Ratios
Project Managers need all the help that they can get, so it is pleasing when we receive requests for specific useful information that might be available from the ISBSG data. One such recent request was:
“What ratios of effort should I allow for each of the types of roles in a software project?”
If you are trying to plan a new project or put together a quote for a customer, or if you want to benchmark the effort ratios of your completed project against other similar projects, this is the sort of information that you need. This prompted an investigation and analysis of the ISBSG data that has resulted in a new Special Report: Planning ProjectsRole Effort Ratios
The ISBSG collects data on roles and effort. We have four levels of effort:
Level 1 Development team
Level 2 Level 1 plus support staff
Level 3 Levels 1&2 plus computer operations
Level 4 All the above plus customer’s effort
In addition we collect effort data for a range of roles, the main ones being: Project Management, Business Analysis, Software Architecture, UI Graphics, QA & Testing, and Programming.
Of course, not everyone gives us all the data that we want, but we found quit enough projects to allow us to examine the effort for various roles and come up with some useful guidelines. In this report we initially provide a general guide to the likely
percentages for each role in a project, but then we look at the impact of special factors. For example, the ratios are different if the project is for an external customer as opposed to an in-house customer; enhancement projects and new development projects differ; the type of programming language and the use of tools also have an impact. In the case of project management, depending upon the characteristics of the project the percentage of the total time that the project manager spends managing the project varies from as low as 7% for in-house developments to 11% for projects for external customers. That will make a big difference to your project plan or quotation.
eNews Volume 5 Number 2 April 2006
Early Lifecycle Estimation
We have updated the Comparative Estimating Tool and made it available as a
stand-alone product. Now you can:
eNews Volume 5 Number 1 March 2006
Helping Project Managers
Intuition and experience helps project managers recognise danger signs and "know" the things that are likely to have an impact on their projects. The ISBSG project history sometimes reveals issues that have previously not been given much thought, but often the ISBSG data provides solid evidence to support that hard earned experience and intuition.
Recently, in response to a question from one of our customers we looked carefully at 15 C# projects in the Java & C# data subset to seek answers as to why their productivity varied so greatly. We discovered some factors that we thought would contribute to the wide range of productivity values. Although this is a very small dataset, many of these characteristics will apply to any software development project, irrespective of the programming language. They can serve as a useful reminder of some of the project characteristics that should be taken into account when estimating projects or benchmarking their performance.
You can download a free copy of this paper from the first news item on the home page of http://www.isbsg.org
The second news item provides information on the Java & C# data subset.
ISBSG Benchmarking Standard
The ISBSG has produced a draft standard for software benchmarking. This is based on the ISO/IEC Standard 15939 Measurement Process: 2002. It has therefore adopted the structure and major activities described within ISO/IEC 15939:2002 adapted to the specific needs of a benchmarking process. A draft version of the document is available for you to try, review and offer comment for improvement. You can download this draft from here
eNews Volume 4 Number 8 September 2005
Practical ways to use the ISBSG data
The ISBSG software project history data is used for many purposes. A new paper, which is available free from here describes some of the practical ways to use the data, including to:
- Estimate software project size, effort, cost, duration
- Verify completeness of software requirements
- Be aware of project risk by checking the reality of estimates
- Acquire custom-built software on a price per functional unit basis
- Determine an appropriate project team size
- Manage the progress of projects
- Benchmark project performance
- Plan software development infrastructure
This paper provides practical examples of how to use the ISBSG data, and charts & tables produced by the ISBSG. You can download a free copy of this paper from the ISBSG home page: http://www.isbsg.org
ISBSG books on-line
The ISBSG books can be viewed on-line at this link. Register for a free subscription trial of the ITPro service. This will
enable you to look at some of the content of the Estimation, Benchmark and Compendium books.
ISBSG Subscriptions
To learn about the benefits of subscribing to the ISBSG visit this link
eNews Volume 4 Number 7 **September 2005
Estimating software projects
Estimating how much a software development project will cost and how long it will take is quite obviously not an easy task. The industry's track record speaks for itself. But perhaps we are making it harder than it needs to be. If you think of what is involved in estimating the cost and duration of building a custom designed house you can gain some useful pointers. Initially you probably only have some broad characteristics of the house that you want; single or double storey, numbers of room types
and perhaps construction material. An architect or experienced builder will use these to estimate a likely size in square meters and will then produce a cost and duration range which you can use to decide whether your concept is feasible.
If you decide to progress to the next stage your requirements are defined and drawn up into a set of plans that can then be used to provide you with a fixed price quotation. Of course there may be interim steps or a number of iterations of the plans, each of which can provide a more refined estimate. The more detail you can convey about your proposed house the more refined the estimate, and eventually quotation, that can be provided.
Software is not greatly different. When a software project is simply an idea, you might know the type of project, main requirements, the likely development platform and perhaps the development language. You need to provide an indicative estimate of the possible cost and duration to give an indication of whether the project idea is feasible. If the project progresses beyond the concept you can garner more information and refine the estimate range. Just like the house, the more detail you have, the more accurate an estimate you can provide. There are many tools and techniques that you can use at the various levels of a project's maturity to refine your estimates.
Estimating Tips
It is important to think in terms of a number of maturing estimates for a project. Particularly early in the lifecycle always present an estimate range, not a single figure that can be mistakenly assumed to be accurate. Use at least a couple of techniques to cross check your estimates.
The latest ISBSG Special Report describes an early lifecycle estimating technique that uses a combination of formulae and the ISBSG project delivery rate tables. Using the information in this report you can produce an estimate range for the effort and duration of a proposed software development. Practical examples and the required tables are provided. This report is available free to ISBSG web subscribers or can be purchased.
eNews Volume 4 Number 6 August 2005
Managing the progress of projects
The ISBSG Project Phase Ratios can be used to manage the progress of a project and realign the project estimate based on the known effort taken for the early life-cycle phases. During the life of a project, a project manager can measure how long each of the project phases are taking. If, in the early lifecycle phases, the project is experiencing over or under run, ie if there is a significant difference between the actual effort taken and the estimate, then the ISBSG pie charts of the ratios of the project phases can be used to estimate the likely overall impact on the project.
These charts show the project phase ratios for new development, re-development and enhancement projects calculated from the projects in the ISBSG repository. So, for example, if the planning phase of a new development project has taken 120 hours instead of an estimated 80, then it will be worthwhile to use this actual figure to check the likely effort required for the remaining phases. Using the pie chart; if planning has taken 120 hours and represents 6% of the total project then the following figures can be used as a guide for the effort that might be required for the other phases:
Specification (20%) 400 hours
Build (48%) 960 hours
Testing (17%) 340 hours
Implementation (9%) 180 hours
The project manager can use these as a check against the estimated effort hours for each phase and adjust the figures if necessary. Phase ratio charts are available in The Software Metrics Compendium.
eNews Volume 4 Number 5 July 2005
Development Team Size impacts productivity and delivery
There are three main factors that impact software development productivity and delivery time: programming language, development platform and team size. The development platform actually indicates the development process and
environment. There are differences in development productivity between platforms, the likely reasons for this are the development process used and the business environment. Analysis of the software projects developed on a mainframe platform
environment show that these projects have more business units involved, support larger numbers of users and make more frequent use of methodologies, (particularly methodologies purchased rather than developed in-house). The use of a purchased methodology tends to produce a wide range of documents including specifications, designs, plans, change control, issue reports and test cases, consequently such projects take longer to produce each functional unit of software, (hours per functional
unit). Programming language is pretty straightforward, some languages are more productive than others. The ISBSG provides tables showing Project Delivery Rate, (PDR hours per functional unit), for the many languages represented in its repository of project history. Having got those two locked away you can start to think about your team size. Analysis of the ISBSG data indicates that team size does have an impact on software project productivity, speed of delivery and duration. The likely impact of team size, particularly the negative impact of larger teams, should be considered when planning a project. ISBSG Web subscribers can download a copy of the Special Report: The Impact of Team Size (on Productivity and Delivery) from the Subscribers’ home page. Non-subscribers can purchase the report.
eNews Volume 4 Number 4 June 2005
The lowest quote is rarely the best quote
We receive a lot of questions about how to judge whether a quotation for software development is realistic and reasonable. You will note that I put "realistic" first. You can accept a quotation for a software project that is extremely competitive, but later discover, too late, that it was never realistic and your project is in serious trouble because the developer, either through incompetence or deviousness, understated the effort required. Often an organization needs an initial estimate of the likely cost of a proposed project so that it can decide if the idea is feasible prior to proceeding to detailed definition and formal quotations. In this eNews we will look at ways of verifying both estimates and quotations. If you don't have skilled and experienced people in-house to apply these ideas contact your country's software metrics organisation The following ideas are specific to the software component of a project, other aspects such as hardware requirements, user training etc should be verified separately.
Initial Software Estimates
You can reduce your risk and improve your evaluation of software estimates, (either outsourced or in-house). The steps are quite simple:
- Estimate the size of your project (explained below)
- Define the project characteristics (language, platform, business type etc)
- Reference ISBSG tables for "average" productivity hrs for your type of project
- Multiply the size by the "average" hours then by a reasonable hourly rate.
- Compare the result to the quotes that you have received.
If any of the quotes are significantly lower than the result that you came up with then you should be seriously questioning the developers as to why they think they can perform so much better than industry "best practice".
Let's look at an example:
1. Estimate the size of your project
You don't have to do a full functional size count to estimate the functional size of your project. There are many size estimation
techniques some of which are explained at ISBSG or in more detail in the book: Practical Project Estimation 2nd edition
For this example let's assume that you know from your logical data model that the number of internal logical files is 40. Using the model for the early prediction of functional size from the ISBSG web site you can quickly calculate a rough software size of 1,250 function points. Of course you should work on a range of + or20%, so let's say between 1,000 to 1,750 function points.
2. Define the project characteristics
For this example we will assume that the basic characteristics are that the development will be multi-platform and written in Java. By referencing the tables in the ISBSG Practical Project Estimation 2nd edition we can establish that the median delivery rate for projects with these characteristics is ~6.5 hours per function point, (or you can use the range each side of the median of between 6.0 and 7.5 hours per function point). Using our size estimate of 1,250fp multiplied by 6.5 hours per function
point, we come up with 8,125 hours of effort for the development. Multiply that by an expected hourly rate of say $80 resulting in an estimated cost of ~$650,000. Now compare this to the estimates that you receive. An estimate of $400,000 might not be the bargain that it first appears to be. Perhaps the developer is incompetent or inexperienced and has only guessed at a
figure. Perhaps the developer is unethical and is quite aware of the likely true cost but submits an unrealistic estimate in the hope of securing the business and increasing revenue through variations and additions. By doing this small amount of preparation you are able to question the estimates received and sort out the rogues and vagabonds!
Software Quotations
You can use a similar approach to verify the veracity of software quotations. Some leading organisations have moved to an approach that requests a quotation based on a dollar amount per function point. The steps are as follows:
- Complete a detailed functional specification
- Have your software sized by a specialist
- Issue your requests for quotations asking for a dollar amount per function point.
- Use the ISBSG tables to establish the industry "best practice" median productivity hours for your type of project (eg hours per function point for multi-platform and Java)
- Compare the quotations received to the range obtained from the ISBSG tables.
As with the Estimate example above, this approach allows you to verify the quotations that you have received and remove the unrealistic ones, whether too high or too low. By working on a dollar per function point basis you have also gained control of the cost of any additions or amendments that arise during the development. There is a lot of information available to help you better manage your IT projects. You don't have to fly blind any more.
eNews Volume 4 Number 3 April 2005
Package Based Projects
The latest ISBSG special analysis reveals that choosing and implementing a package has advantages over developing new software, provided that the implementation is either turnkey or utilises customistation facilities provided with the package. Package projects that involve changes to the package source code perform worse than development projects. The full report is available.
Estimating Functional Size
We receive quite a lot of questions about estimating software size early in the life cycle of a project. You don't have to do a function point count to estimate the functional size of a project. The ISBSG web site provides some basic information about estimating size. In addition the recently released Practical Project Estimation book 2nd edition has a greatly expanded chapter on size estimation.
eNews Volume 4 Number 2 March 2005
Maintenance & Support Data
The ISBSG has established a Repository of Maintenance & Support application data. Like the ISBSG Repository of Development data, as the M&S Repository grows we will be able to provide useful analyses to assist practitioners who are
responsible for M&S activities. The ISBSG has produced an initial analysis report based on the application data
in the fledgling repsitory. You can download a free copy of this report.
eNews Volume 4 Number 1 January 2005
Software EstimatesHow accurate are they?
The ISBSG has just released a Special Analysis of software project estimates. We look at estimates of size, effort, duration and cost; how people have gone about estimating their projects; the accuracy of the estimates and the relationships between
estimates. Here are some of the findings:
- Size estimates are usually based on a data model; functional specification or analogy
with a previous project.
- Project effort estimates are only accurate for less than a quarter of projects
- Despite effort being poorly estimated 51% of projects were delivered on time
- When functional size-based techniques are used for a cost estimate, the estimate is within 20% of the actual cost 90% of the time
- When newer technologies are involved, estimates are less accurate
This paper should help project managers to better understand the factors that impact the accuracy of software project estimates. Some of the questions that its answers are
- What techniques produce the most accurate estimates?
- Is project size a factor in estimation accuracy?
- What are typical over-runs and under-runs ?
- What types of projects are most likely to be poorly estimated?
- What is the most reliable basis for an estimate?
The full report is available
eNews Volume 3 Number 5 June 2004
New data release
The latest release of the Estimating, Benchmarking & Research CD (R9) has 50% more projects, (now over 3,000), which reveal some interesting characteristics:
- RAD/JAD techniques are used in 25% of the projects
- OO techniques are used in 15% of the projects
- Prototyping is used in 26% of the projects
- Data modelling is used in 58% of the projects
- There has been an influx of Java, C++ and C projects.
You can download the Demographic report for the data from www.isbsg.org Select Downloads from the vertical menu.
For a full description of the data and estimation tools contained on the CD, select Benchmarking > Benchmarking Data CD.
If you purchased Release 8 in the last 6 months you will receive a special upgrade offer via email.
You can buy the Estimating, Benchmarking and Research Suite data CD
You can buy The Benchmark Release 8
eNews Volume 3 Number 4 June 2004
CMMI and the ISBSG
The Capability Maturity Model® Integration[1], (CMMI), has gained acceptance as a guide for software organisations that want to gain control of their software development and maintenance processes and mature toward software engineering and management excellence.
The Capability Maturity Model defines five levels of software process maturity from Performed, (no stable environment), through to Optimizing, (focused on continuous process improvement). Most small or medium sized organisations will typically set their sights on Levels 2 and 3. Level 2 is the Managed Level where policies for maintaining policies and procedures for software project management are established. Level 3 is the Defined Level where a standard process for developing and maintaining software is documented and used.
Within the requirements of both levels 2 and 3 are activities that cover Size Estimation, Effort & Cost Estimation, Software Scheduling, Project Data Collection and the use of Project History Data. There are a number of references to using historical data to help in estimating project effort, cost and duration. The ISBSG data will meet this need where an organisation does not have its own history data or where it wishes to supplement that data.
Where CMMI seeks documented procedures for project estimation, the ISBSG Practical Project Estimation book provides ready made estimation procedures for three macro estimation approaches.
For any organisation that is striving to move up the CMMI model, use of the ISBSG data and publications will help.
CMMI references where use of the ISBSG data and estimation processes could help to achieve compliance
Project Planning GP 2.2 SP1.2, 1.4, 2.3, 3.2
Measurement & Analysis SP 1.1, 1.4, 2.3
Project Monitoring & Control SP 1.4
Integrated Project Management SP 1.2, 1.5
Organisational Process Focus SP 2.4
[1] http//www.sei.cmu.edu/cmmi/cmmi.html