Comments on: All Over the Map — The Imprecision of 24 California HERS Ratings

By: Andrew Lesperance

Andrew Lesperance — Thu, 25 Sep 2014 17:34:47 +0000

The problem is absolutely
The problem is absolutely with the raters and the QAD, but I think the problem is solved at the Provider level.

We have the same problem here in Michigan. We have a rating company that does business in our area from Indiana. Some of their builder clients have switched over to our company due issues they have had with them. We have reviewed the ratings they have done and they are basically rubber stamping all their ratings. The builders have told us that their raters spend about 10 minutes at the house at final inspection and in most cases don't even do a pre-drywall inspection. This is extremely frustrating to know there is a rating company from out of state rubber stamping projects and taking away business from our company.

If this type of thing continues to happen the HERS rating will lose credibility. Bottom line is we need better QAD and providers.

Its good to know that there are providers out there like EnergyVanguard that look at more than just the 10%. We work with Building Efficiency Resources (The BER) and they are a great Provider as well. They look at all the ratings that come through. I truly believe that if we want to solve this problem it gets solved at the Provider level. RESNET needs to get the shady Providers out of the system. Not only does this improve QA naturally, but it gives an opportunity for the good Providers to get more business.

As a rater, I understand that there is going to be variation among raters. The process isn't perfect. The raters that are rubber stamping projects are what we need to go after as an industry. They will ultimately bring us down.

By: Ted

Ted — Thu, 18 Sep 2014 18:30:27 +0000

New homes don’t have energy
New homes don’t have energy histories, so solving variance issues will be difficult (though Danny Parker believes this is solvable; http://bit.ly/dannyparkeraccuracyvideo)

Houses with existing histories there is an obvious reconciliation point. That RESNET doesn’t connect theory to reality here is disappointing.

Others have raised the issue of HERS being a silk suit:

http://bit.ly/energywise-hersratingsdontcorrelatetoenergyuse 

http://bit.ly/HERS-2PerformanceFAIL 

Will Allison be the lightning rod that drives change and accountability into this broken system? We can hope and admire his effort, but I’m wouldn’t bet the farm.

By: John Proctor

John Proctor — Wed, 17 Sep 2014 03:58:15 +0000

Several of the questions
Several of the questions above should be answered.
1) Most if not all of the Raters were experienced. I know some of them myself.
2) The houses were operated with the same set point schedule and occupancy internal gains as are integrated into the HERS software.
3) So the only "true up" as adjusting to the Standard outdoor conditions that are used in the HERS weather file. This too was done.
4) Correct, this shows two problems -- data collection and input AND fundamental errors in the software.

By: Sean @ SLS

Sean @ SLS — Wed, 17 Sep 2014 01:17:03 +0000

I got to say I have to wonder
I got to say I have to wonder about the "who" was hired and their experience. As for simulating use - that's fine as long as it meets how it is figured for RESNET or they should calculate out the difference to true their model if they wish to do that

By: Andrew McDowell

Andrew McDowell — Tue, 16 Sep 2014 23:46:09 +0000

Thanks for that Allison. I
Thanks for that Allison. I am curious if only the Blower Door and Duct Leakage test results for each rater could be posted, to see what variance exists in the measurements? And also, were the models "trued" to actual energy consumption history using the Audit tab (or whatever the California software equivalent is)? In other words, was the intent of the survey to assess ratings, or to assess audits?
Thanks for posting.

By: Nate Adams

Nate Adams — Tue, 16 Sep 2014 23:42:16 +0000

I was frankly very
I was frankly very disappointed with the HERS rating I got for the Hiram College TREE House. It was effectively useless. What does it mean? The data contained in it was very difficult to decipher.

... And now I'm finding that there is huge variance in this rating that I already couldn't find meaning in? Hmm. I only selected a HERS rating so they would have something to brag about for the project, what does it actually mean, though? And how can I explain to them that the number will change when a new energy code comes out?

Wouldn't usage/sf or annual $ usage be more useful, a the end of the day?

Also, you said "I've talked with enough people to know they have some of the same problems we have in the rest of the country with poor QA of raters by providers and insufficient oversight of providers."

Isn't that a systemic problem? Could the design of the system be flawed? What reason to raters have to be accurate if their work is never tied to anything?

Sorry, I've been cooling to the idea of HERS ratings since the TREE House, and this is not helping. I just don't see the value if there is no accountability for raters.

By: John Nicholas

John Nicholas — Tue, 16 Sep 2014 23:38:52 +0000

This problem is directly
This problem is directly related to the Flow of Information from RESNET to QA designees and a separate flow from RESNET to Raters and a 3rd stream from a QAD to the raters involved.

If RESNET were to communicate the same information to all QADs and RATERs on documentation requirements.

My QAD requires a ruler in the insulation picture to be in my file. If every one of those 6 raters were required to have the same information in their file for documentation, any discrepancy in the attic insulation could be dealt with.

If the transparency in communication was present, some mistakes could be eliminated. The other piece would be for RESNET to select files for the QAD to do a desk audit. Provide all required documentation. If the RATER didn't collect what both the RATER and the QAD had been told to collect ... RESNET can take action.

Takes 30 days to implement.

By: w d

w d — Tue, 16 Sep 2014 23:10:34 +0000

The surveys in this study are
The surveys in this study are ALL predicting electrical usage for cooling above actual. There could be several causes for this but it's interesting that the predictions ALWAYS have a positive variance. Not a single data point below actual usage. In troubleshooting parlance that's called "PECULIAR".

There are other issues but that's the one that jumps out at me.

Allison, Kudos for publishing troublesome work in progress.

By: Bob Ellenberg

Bob Ellenberg — Tue, 16 Sep 2014 22:36:31 +0000

It appears to be a vicious
It appears to be a vicious cycle of trying to get people of competence and integrity. I think we only needed the Raters to monitor what the building contractors were NOT doing properly and now we need monitors for the monitors. Pay for a competent conscientious contractor and get what you pay for.

By: Allison Bailes

Allison Bailes — Tue, 16 Sep 2014 22:24:05 +0000

Nate A.:
Nate A.: Good question. I haven't seen all the data from their study, but what I have seen indicates some serious problems with data collection and entry. Ceiling insulation R-values that vary from 19 to 49 in one house is a rater problem, not system design. Also, I don't have direct experience with ratings in California, but I've talked with enough people to know they have some of the same problems we have in the rest of the country with poor QA of raters by providers and insufficient oversight of providers.