Our data blog

Posts about live events information, its quirks and other data related to our work


The uses of live events data

9 November 2018

The List has always, to some extent, been a data business. Back in 1985 we manually copied live event listings from printed venue programmes in order to publish them. We wanted to help readers of our fortnightly magazine, as we put it back in those days, Get A Life. Even then we were assembling data. Ten years ago, we collected around 3,000 event listings for Edinburgh and Glasgow. These days of course the world has changed. These days we use AI and semantic in processes for assembling the UK’s most comprehensive set of live events data – around 40,000 event listings and an average of 500,000 performances.

And as that world has changed, so has the number of uses to which we can put our data.

Our very first information services clients were media companies. We used to supply local newspapers with print ready copy of curated lists of events – ten of the best clubs or the best gigs to go to that week. Over time many of those print contracts became digital ones.

We also work with many destination marketing organisations, such as PeopleMakeGlasgow.com, supplying them with around 20,000 performances of live events at any one time which shows off the deep and rich cultural offering that the city provides. The data provides cities, regions and countries with competitive advantage in a fiercely contested global tourism industry (where experiential tourism is on the rise).

But we now conceive of a much wider range of services.

Looking at accommodation as a single vertical, live events data can provide business intelligence around future demand – essential for accurate yield management. It can also provide services around what to see and do close to the hotel for an authentic visitor experience, delivered by email pre-visit or even Alexa voice-based concierges in real time. As hotels try to wrest their audience back from online travel agents there is a need to improve their own CRM and an understanding of what the guest has experienced contributes towards improved efficiency of future messages. After all – if you liked the craft beer festival you experienced in Manchester, wouldn’t you also like to hear about the six best craft beer festivals coming up across the UK (that just happen to be close to a hotel with a personal discount in place)?

As our cities become smarter, so their need for data increases.

Our live events data will be used in two forthcoming projects in our home city of Edinburgh. Firstly, in refuse collection where our past data can be tallied with bin capacity monitors identifying which events cause the most rubbish. The results can be used to predict where refuse collection services are required in advance leading to more accurate planning rotas and more efficient collection.

Secondly, we plan to work with Transport for Edinburgh looking at the effect that the Edinburgh Festival, the world’s largest arts festival, with collectively over 4,000 events and more than 100,000 performances in a month, has on the cities transport network. Again, the results can be used to predict transport usage more accurately, avoid bottlenecks and make sure that both tourists and locals alike have a more pleasant experience in August.

We are also planning a hackathon with the emergency services to help better determine where forces should be placed on a Friday and Saturday night based around the movement of people and how many doctors are required by the NHS of an evening.

These are just some of the ideas we are working on. We are seeing the development community come up with plenty more that we never even conceived of, which is why our API is a free to play sandbox up to a certain volume of requests.

The List in fact is endless. More strategic positioning of taxis based on events that are just about to finish. Correct staffing and food levels for restaurants based on increased localised demand. SatNav advice that takes future traffic into account based on specific events.

In addition to all the business intelligence that live events data can provide, we can also help to inspire people to go and visit an event – to travel by rail or air and go see something amazing. Ticketmaster still state that the biggest reason still that someone doesn’t go and see a show is because they didn’t know about it(Mobile world congress, 2016).

The syndication of our data far and wide by a multitude of businesses and organisations will help to solve this problem. Which all ties rather nicely back to 1985 where the business set out with an ethos to provide the best arts and entertainmnt information to the widest number of people possible. At one time that was just Edinburgh and Glasgow with a magazine.

In the future, we hope to do this for the world.


The State of Arts criticism across the Edinburgh Festivals – some data

19 September 2018

Edinbrru phonetically, Edinborough in a North American transatlantic drawl, or Auld Reekie or Old Smoky, in times gone by, hosts the world’s largest set of festivals taking place all round the year. They come to a peak in late July and August with a flush of 7, 8 or 9 festivals depending on how you count.

Across the core August festivals, namely the Edinburgh International Festival, Edinburgh International Book Festival, Edinburgh Art Festival and Edinburgh Festival Fringe and then a set of overlapping free festivals, there are nearly 4,000 shows with 50,000 performances at one of 400 venues.

And all in a city of just half a million. With the same number of visitors on top, the city is bursting with people.

With so many shows the role of the critic becomes as important in this context as for any Broadway or West End show.

And with so many new shows, and so many new performers, they often can't succeed in getting the audience of which they hope, so to get reviewed at all is an ambition, to get reviewed by well-known titles more so, and a good review is itself the dream. Not only may a good review fill the show in Edinburgh, but can also for subsequent runs over months and years to come.

So, it does matter to the audience and the performers if there is a decline in the extent of reviewing at the Edinburgh Festivals.

The List have aggregated reviews across a range of titles since 2009, and publish this data.

For a presentation in August 2018 organised by the Edinburgh Festival Fringe, entitled The State of Arts Criticism, we took a consistent set of titles that undertake Edinburgh Festival reviewing and looked at the decline from 2012 to 2017. Our data shows a significant decline in the set, from 5134 reviews in 2012 to 3169 in 2017.

The dotted lines represent titles which, following enquiry with them, we understand do not pay their reviewers.

Alongside this decline, we also examined the rigour with which journalists approached their task.

With so many shows it would be healthy and wise for reviewers and their editors to select shows that are expected to be the more interesting. But the outcome should be balance none the less, of poor shows and good shows. And the star rating system can illustrate the balance that is sought by displaying star ratings that conform to the normal curve.

So, it is interesting to see that titles that employ paid critics, in aggregate, deliver a result that conforms more closely to the statisticians ideal. A perfect normal curve would have a peak at 2.5 stars and a broad curve. So the further to the right is the peak, the greater the star rating inflation.(The Scotsman just looks an outlier as reviews under 3 stars are usually only printed.)

Comparing critics and users ratings shows that the later mostly undertake the task of reviewing as a promotional activity.

Our final chart shows that The List published more reviews online in 2018 than any other title.

We have more data covering other titles and by writer which can show some interesting trends. And we hope may be the subject of a later post.


Starting out, and arriving at an event taxonomy

31 August 2018

The List started collecting live event listings in 1985. We possess the UK’s leading set of live events data with up to 600,000 future performances of 45,000 events at one of 85,000 venues across the length and breadth of the UK.

The events are spread across a range of event types and a while ago we held an engineering away day during which we developed a taxonomy of event types, which I describe in this post.

Before embarking on the post, and by way of background I wanted to quickly outline our history as a listings collector and an overview of our means for sourcing this highly fragmented data from a very large number of venues. Over the decades we have worked our way steadily up a hierarchy of methods for sourcing listings –

  • 1980’s - postal mail, card index files
  • 1990’s - fax, email and manual input to db
  • 2000’s - web sites, csv and first loading
  • 2010’s - web forms, file transfer

Now our preferred and dominant input is structured files from box offices and other primary sources.

We receive and process 200 or more regular feeds, many daily, be that from cinema chains like Cineworld, theatre groups like Ambassadors Theatre Group ATG or an individual art centre, such as Stockton Arts Centre ARC.

This activity is supplemented by data we receive on our web form, with around 40,000 submissions in a year, which allows us to accrete a big range of smaller events from organisations that don’t have the ability to supply the structured data. All such submissions are subject to human oversight.

We target over 99.99% accuracy in core data of date, place and time.

So, looking across the range of events information we receive, we see events falling into 9 categories which we outline in the table below:

Event type Category Examples Deterministic features Possible overlap into
Consumer 1 Sale at Harrods, craft fair, car boot sale, celebrity appearance Commercial, transactional Entertainment
Business and professional 2 AGM for BP, sessions of parliament, public lecture by Dir of Institute of Chartered Accountants Open to public, possibly some qualification to attend, free or moderate entry price Community
Community 3 Public Info Notices of planning meetings, blood donor locations, marches, church fetes Of interest to narrow geographic area, or well defined community of interest Business
Conference, expo 4 Crufts, Wedding World, Edinburgh International TV Festival Trade show with stands, speaker sessions, sometimes significant entry cost Consumer, business
Entertainment 5 The original and primary List categories including theatre, music, dance, visual art etc. Consumer
Education, workshops, courses and classes 6 Rock Choirs, Yoga classes Led by a teacher Activities
Activities, physical recreation and participatory sport 7 Wildlife Trust walks, London Marathon Participatory Education
Attractions 8 Legoland, Albert Memorial, Green Park, British Library A range of places of interest that may be open 365×24, have daily or seasonal opening dates and times. May have timed entry tickets.
Restricted (generally, not for publication) 9 Private views, an employee only Christmas club night, press previews and photoshoots Invitation private views, employees only, personal events Business and professional

By background, The List’s consumer publishing activities have by necessity been focused on events open to the general public, and given our remit to reach a broad audience, we have not focused on Category 4 events.

Given our strong commitment to a business model under which all listings are free, we also need to manage some cases of those promoting category 1 activities, who endeavour to position them as events and gain listings, when really promotion is what they seek for their commercial activities. But we have a soft spot for car boot sales, so Category 1 are certainly accepted.

As the delivery of listings via our api, expands through a widening set of channels, including smart cities, hotel groups and transport operators, and well beyond our earlier client base dominated by the media sector, then the value in including events from all categories above and especially Category 4 increases.

We are fortunate to have had a 30+ year history of gathering, enriching, managing, organising and distributing live events listings in a standard event structure, but we remain very open to adding to our understanding of this data space.