Point Topic has been involved in internet and broadband mapping since before there was very much broadband. Our first datapoints in 1998 counted just over one million broadband subscribers worldwide. 2023 is Point Topic’s 25th anniversary!
Mapping in the late 1990’s and early 2000’s was only just emerging from the mainframes. Computationally heavy (for the time) and with limited sources it was a specialist skill. No all-in-one Python packages or ‘Export as: whatever-you-want’.
There were plenty of early pioneers and one of the earliest was John S Quarterman in the US.
US state level and reporting on hosts and aggregating for major clusters (up to 1M in some places!) was a big leap in granularity, the internet hit an inflection point at the end of the millennium as ‘broadband’ started to appear.
It took us a while to get from pure data outputs to integrating and working with GIS (geographical information systems) but by 2001 we had some models running.
Focused at first on the UK, not least because with one million seven hundred postcodes it still has one of the most granular reference systems of any country. Combine that with a few fields and we were processing over ten million datapoints for each output. That was ‘big’ data twenty years ago. It was still a major challenge to get input data with geo-references and often we resorted to tracing maps by hand.
Some early attempts to corral this data were generally manual and still very resource intensive, often relying on handcrafted maps and very little machine help when it came to cleaning and organising the inputs. Even working with partners, like Gavurin, we struggled to make useable portals that we could update easily.
Building on all this experience (and the ‘failures’) we have helped to reveal increasing detail across the UK and Europe.
Working from our desks in offices we generated some of the first and most detailed data and maps of broadband infrastructure and operators, all of which still continues through today… although now it’s much more often from desks at home and with a range of other research options and inputs all enabled by the internet and more powerful parallel processors.
How our European journey started
Maps based on satellite imagery have been key to developing a better understanding of the human geography of Europe. We used those as the basis for our early Europe wide modelling.
CORINE is an example of open source data that enables better analysis and outcomes across a range of sectors. We used it as a source of population densities, and given it being open source, were able to discuss and share our work in a common, granular framework.
With our experience in the UK we were able to put together a successful bid for some European Commission projects looking at pricing and adoption across Europe and that led on to the first version of our technology and operator presence mapping and accompanying forecasts ‘Broadband in EU Regions via satellite’.
The original website has gone, tears in the rain, but BEREC is still working hard though to get European internal telecoms market integrated.
The project helped us do more work across Europe including some interesting times during the Arab spring when the provision of satellite data and broadband coverage across the Mediterranean basin was a priority and just try and get the Mauritanian Telecoms Regulator on the telephone! Nonetheless the drive for full adoption had started and we helped scope out who was going to be well served and who might be left behind.
Next Steps in the EC - two forward, one back
During this time we have seen excellent resources emerge as member states and the rest of Europe woke up to the importance of broadband for their economies and their populations. The market, already aware, was actively capitalising on the lucrative and high-revenue opportunities as regulations relaxed and competition intensified.
What has not happened yet to any satisfactory degree is actually publishing an interactive Europe-wide map. I sat on a committee for a number of years that eventually produced this - https://www.broadband-mapping.eu/ - still the high water mark when it comes to a collective effort and a disappointing and apparently dormant one at that.
What exists today in public sources, (Broadband Coverage in Europe is one) is far from perfect but with an uninterrupted time series from 2010 on, there are datasets available for the spread and impact of broadband across a decade when ‘superfast’ broadband and most of the online population today came to the internet.
Most individual EC member states have progressed significantly in their own efforts however. Online data portals, maps, query engines and regular updates are now commonly published and offer a rich basis for national analysis. Datasets and platforms for international, comparative analysis are still less detailed, less frequent and less geographically granular than the data allows.
Granularity is key - dissolving the boundaries and zooming in
Today in the UK, we offer monthly outputs tracking broadband by technology, operator, ISP, tariff, adoption, churn and more generating around 100 million datapoints a month. Achieving the same across Europe is a long term goal and, although a challenge, we have a good start on it.
One set of challenges when it comes to international statistics, important if the EC is going to steer central subsidies, are definitions. Some key questions, some general and some particular to broadband, that remain difficult to resolve across the EC:
What actually is a household and how many are there?
What is ‘rural’ and what counts as a ‘village’ or ‘town’?
What is ‘superfast’, ‘ultrafast’ and what’s next?
What does FTTH/P/B/S/N/X mean and are they split out in the statistics from the NRA?
What counts as a premises passed by broadband infrastructure? A PoP within 200 metres? 100? 5? Ready for service or still waiting to be lit?
It may seem strange that such basic metrics aren’t set in stone but the more you dig into international, national and even further down to very local outputs and measures the more you appreciate the complexity that exists.
You also become aware (dimly at first but with increasing clarity and not a little dread) of the extent of the compromises and assumptions that underpin so much that we ‘know’.
Agreeing (or imposing) single definitions at a granular level that can encompass and reflect the local variations (and definitions) has to take place to enable coherent analysis. While this isn’t politically possible for the EC we carry no such burden.
So we are able to align national reporting on our systems and enable comparison and analysis across national boundaries. Dissolving national boundaries, at least in terms of definitions, is key to a single European market.
Proprietary versus open data
There are plenty of reasons why mapping infrastructure in detail (and making it public) can be problematic. Reasons are commercial, political and even security-related, as telecoms is a vital national resource and historically single points of failure made networks and exchanges top targets. The internet itself, with the advent of packet switching in particular, has at least made that threat less real.
All of these have slowed down the development and availability of datasets that span more than small areas and brief moments in time. The recent efforts in the US, their first update in years, (https://broadbandmap.fcc.gov/home), and the ongoing issues with the FCC itself have highlighted the commercial impact on ‘open’ data as they struggle to produce their own, accurate, map. Given the extent of lobbying in that market, there is clearly also politics at play here, as internet players and providers are among top donors to US politicians and keen to preserve any competitive edge they can.
The cat is (almost) out of the bag in Europe though. Countries have premises level query sites for public use, usually a good interactive map, as well as supporting access to digital mapping (and permit and planning applications) for operators and regular reports and submissions to the National Regulator and via them to the European Commission (Broadband Coverage in Europe).
There is also another concern when it comes to data. It is possible to zoom in too far and by doing so invade or compromise the privacy of the consumers we are trying to support. Gathering and analysing data at premises or even personal level is possible but it is also invasive and in the worst cases illegal.
While Point Topic does process some personal data, mostly in the form of speed tests, we do not report or sell personal data. For fixed broadband you generally do not need that level of insight and it makes life a lot simpler for us (and our clients) when there are no issues in the data stacks when it comes to infringing privacy or the rights of individuals.
How far have we come and our next challenges
Starting with country level analysis and thirty one rows covering the UK, the European Union, Switzerland, Iceland and Norway we have now progressed to overlaying a square kilometre grid with four million rows blanketing the land area of Europe.
Technology, operator, adoption and pricing are all available from our datasets and to help us with extending and applying those datasets we have built a new platform.
All data and particularly models have to be viewed as indicators (you don’t want to know how laws, sausages or national statistics are made) and margins of error must be understood along the chain and in the outcome and where statistics are most abused in their interpretation.
Normalising inputs while keeping a path to the source is key to providing useable outputs that can nonetheless retain a relationship to the original data. A large amount of the work we do at Point Topic and at our new European sister company, Expert Intelligence, is building and maintaining the tools that help us do this.
Our first outputs are at NUTS3 level…for the moment.
With Expert Intelligence we have launched a distillation of our experience and data in Europe. Based in Portugal, Expert Intelligence will specialise in market research and granular broadband mapping across the mainland, using the expertise of the UK parent company to develop and extend the same data-led approach as a new world of machine learning and intelligent agent (guided by tech and sector experts) enabled research and analysis emerges.
We have refreshed and updated the European Broadband Markets product that Point Topic built over the last decade. Access to more data, better compute time and costs, and a move from spreadsheets to databases have allowed us to launch Expert Intelligence with a valuable timeline of broadband technologies, speeds, operators, costs and adoption across Europe.
Across the mainland, our aim is to continue and extend Point Topic’s work providing comprehensive, pan-European datasets that cover the last decade and on into the 2020s.
 |  |
Figure 8A: Contrasting pictures - Cable coverage in Germany | 8B: ... and FTTP Coverage. Source: Expert Intelligence/Point Topic |
You can see more about the European Broadband Markets on the Expert Intelligence site.
The next decade – supporting analysis across Europe
There are due to be fundamental shifts in the European telecoms market in the coming years.
The ’Gigabit Infrastructure Act’ is making its way through BEREC and the EC and the consultation period for ‘The future of the electronic communications sector and its infrastructure’ has just closed and change is coming. Targets and expectations are being ramped up, and ‘very fast’ broadband is no longer 30Mbps, but 100Mbps or 1Gbps. IRIS2 might change the rules of the game again, with massive amounts of money poured into European-owned LEO constellations to power connectivity, all within the next few years and thanks to Starlink with proven technology.
At Expert Intelligence, our first models will focus on forecasting the extent of fixed line, particularly FTTP, across Europe through to 2030, with scenarios reflecting our analysis of the Gigabit Act and the country by country response, planning and expected outcomes.
Driven by a kilometre grid, the European Demographic and Geographic Analytics Platform will be extended with predictors and metrics and supported with our EBM timeline and decades of experience. From there, the goal will be to go more granular, with more datapoints and more insight.
We will be publishing forecasts soon. For more on what we can offer you and what is coming down the pipe just register on the Expert Intelligence site and we will make sure you are kept up to date.
Oliver Johnson
CEO Point Topic and Expert Intelligence
May 2023
Comments