Small sample of households & information sharing

The Smart Energy Research Lab (SERL) data contains smart meter and contextual data from over 13,000 households. However, not all of these households met the criteria for inclusion in this analysis. For instance, some were excluded because their data did not span the entire analysis period. Ultimately, to ensure the sample remains representative of Great Britain, we focused our analysis on a smaller, refined sample of approximately 6,000 households.

The process of defining the number of energy profiles is a careful balance of data science insights and expert domain knowledge. It's crucial that these profiles are of an appropriate size – not so small that they prevent us from showing meaningful information due to statistical disclosure rules, nor so large that they lose their usefulness. Our current sample size means some profiles are quite small, which can limit our ability to provide granular breakdowns, such as property type distributions within specific profiles. This issue is further enhanced by missing contextual information, given that a proportion of values is unknown for some variables, often requiring us to group unknown categories with known ones due to low counts.

With a larger sample or enhanced synthetic data, we could further refine these profiles to identify ‘true archetypes’ of energy consumption and household characteristics. Making high-fidelity synthetic smart meter data openly available is crucial for leveraging this information to inform the energy transition.

Extracting actionable insights

Households were clustered using consumption data. Sociodemographic and other contextual information are not well clustered, making it hard to identify ‘true archetypes’ and identify meaningful insight. As an example, wealthy households with solar panels and low-income households might cluster together if both have low electricity consumption (although for different reasons).

Information gap for low-income and deprived households

We often have less to say about those on low incomes and in more deprived areas, as they typically don’t have technologies such as heat pumps or solar panels installed in the house.

Inherent bias in data

The data we have access to comes from houses with smart meters; hence, the data is inherently biased to what we can learn from homes with smart meters. Smart meter coverage is still quite low in GB, with the latest statistics showing the number of installations in the UK reached 66% of homes and small businesses, with 39 million smart and advanced meters in operation across Great Britain as of the end of 2024.

You can find detailed information about the underlying methodology in this document.