Hello, you are listening to the Streaming Audio podcast. And one of the things I love about this industry is it can take you anywhere. If you are interested in movies, movies need programmers. Are you interested in eradicating malaria? They need programmers these days. Are you interested in the mating habits of migratory birds? Yeah, even they need computer models. As programmers, we get to go into every industry. Anything we are interested in, there is a job out there.
And for this episode, we are going to go into a world that I never expected to enter, but it turned out to be fascinating. It's the world of table tennis. And when you start to think about it, it does make sense. The sporting world lives on real-time data. Someone scores a point and you've got fractions of a second to update the scoreboard, let the commentators know, the book makers, the people checking the scores on their phone, to update the history books. Sports are made for live streaming data.
And that's something that today's guest, Vatsan, realized when he joined the International Table Tennis Federation. But he joined them right at the height of the pandemic, and COVID had shut down all the major table tennis matches. So as this thing went, he got about an 18-month window to look at the way world table tennis handled data, and just completely reimagined their systems for the 21st century.
We got to record this one live in Singapore, in Vatsan's office. And honestly, I'm not much of a sports person, but I found this really fascinating. So Streaming Audio, is brought to you by Confluent Developer; I'll tell you all about that at the end. But for now, come and peek into an international phenomenon through the eyes of modern technology.
So my guest today is Vatsan, welcome to the show.
Thanks, Kris. Thanks for giving me the opportunity to meet you up and then explain about how Confluent, has helped us.
Oh, cool. It's nice to be here. For the people listening at home, we are on location in Singapore, doing this in your office. Tell us about this place. So you are born out of the International Table Tennis Federation?
Tell me something about that.
So just to give some insights about, for people who don't know about table tennis, or give more information about the firm that I represent. International Table Tennis Federation, in short ITTF, was conceptualized and formed in 1926. It's almost having a 100 years of history now.
Oh wow, the centenary is coming up.
Almost coming up.
And it's the sports governing body. It's like a FIFA for football. So they manage the rules, they govern the sport, they also have member associations. When I say member associations, it's the national associations, or countries who play, who are members of the sports.
And obviously right through these 90 years of their existence, it's been more focused on governance as such. But then they wanted to get that up over ATP or the FIFA to commercialize this, which is exactly when we had the World Table Tennis was born as a commercial entity for ITTF. This was a startup, which was launched in 2021, January.
Not so long ago.
Not long ago. And then the main focus for WTT, is to get the fans to engage more on the fence, to play table tennis, to participate as we see it in other sports. More and more, we are getting fan engagement. And then with table tennis, one of the key stats that we have seen, it's the most played sports in the world.
Exactly. And because if you see the history of football is one, and then it's quite closely followed by table tennis. You could play ping pong. It has its original origin in ping pong where in China. And then you started to have all of those players, even if you have a small table, people play. So exactly how we started to see the opportunity for reaching out to the fan base across the globe. And that is where we are focusing on now, making the sport to reach out to the fans in a way that it is more commercialized in terms of managing the fans' interest, the players' interest, as well as the sport itself.
Yeah. Because it doesn't surprise me at all that there is a governing body for table tennis that's global. But yeah, I can see how that in the modern world people who are really into it, want live sports results and that kind of thing, right?
Exactly. And that was the whole intent. I'm a sports' enthusiast and in fact, my father was a cricket umpire. If not he was on the international level, at least from a state level, he was a umpire. And that kept me interested in sports and we have been a follower of different sports, including tennis, football. And then obviously, I played a bit of table tennis, but I also a cricketer as well.
And that gave me the interest over to be affiliated to sports somehow. And when I got the opportunity to be part of a legacy, to be part of a sports federation, to contribute to sports, I thought, "Yes, I certainly have to take my chances here."
I know you said in 2020, you were working for the Inland Revenue Service for Singapore?
Exactly. This was the Singapore government.
Probably less exciting?
I would say this as a government organization, we do have a set plan obviously. And it takes its own course in terms of implementation and project timelines and approvals. Yes, we do need those observations in terms of a government organization. Without government organization, we aren't going to survive; we need those. But then much less interesting compared to where I am now in terms of the sports.
Tell me from the start. So where were they when they contacted you? What was their IT set up?
I think from ITTF, to be honest, I was the first IT employee purely focusing on technology. And I was the first recruit for WTT in the technology side as well. And unfortunately, when it comes to pandemic, I'll come to it later. But then when I started the systems were pretty much isolated. We had systems over for different sports products managed in Excel and different situations. So this was just moving on. We had products which were not integrated. This was another bigger pain point that we had as well.
So the first part of it for me was to start off on a digital transformation journey. And my senior management had full faith and confidence in moving over this huge journey, because we had to consolidate historical data close to 90 years. It's the first part of it is the data consolidation. That was the first part that I started to map out. What are the different sources of data?
So the historical data was more important initially than gathering the matches playing today?
Yeah. So to start with, for table tennis, the stats, again, this is the first part, creating the structure of the data. It comes from whether it is a match related data or it is coming from the players' database, or when you are talking about entries or events, the winners, the champions, all of the data had to be consolidated to put in place a structure which we can take it forward. Consolidated structure that in a more refined way that we can move ahead. And that was in the first part of the journey, where consolidate plays a framework over and then start to move on in terms of getting the digital transformation, having a consolidated and integrated suit of systems.
Yeah. So you got 90 years worth of data to build a strong data model end?
Exactly. And that data model was the key to start our journey. Because otherwise it will be like we are talking about having the single source of truth, and that is exactly nicely tied with your Confluence concept as well. Having the same, and maintaining, and managing and handling that single source of truth.
It's perfect fit for an immutable historical logo sector.
Which is exactly why we wanted to get into the Confluent platform. And one of the other things that we also... I had an experience when I was evaluating product, when I was an enterprise architect working with Iris, I was introduced to different messaging platforms. And part of it was evaluating RabbitMQ, and Service Bus, and all these. And that is when I had a chance to meet up with your team, Confluent came in, they had their product. And then they helped us to evaluate for the government, which is where my journey started with Confluent.
Okay. So at the IRS you got into Kafka, and started-
The first time that introduced, I know we had the Apache Kafka, I know the challenges of maintaining and managing the Apache platform as not a asset managed instance. And when I got to know about the features, even at the time, two years, three years back, Confluent was well established. And I was quite impressed at the time when I was introduced to the product, the features, the roadmap that they had for the last next three, four years. That was really impressed me. That is exactly why the moment that I started out the digital transformation journey in WTT, the first vendor that I wanted to look into, the first product that I wanted to use was Confluent. And then because I'm of my experience with working with Microsoft, and Azure, was also one of the hosting platform that I'm using along with data services.
Okay. So your Confluent Cloud on Azure?
Okay. You are happy with the providers, this is a good thing. But let's get more into the technology, right? So you establish this long topic of historical event data. Is that a product in itself for WTT?
No, so the first part of it was... And as I mentioned, you have the players' database, and when you are looking at it, we have 90 years of data even for consolidating the players. Players, then you have events. We have data even going back to 1930 World Championship. That was the first time that we had gone. We have touched the tip of the iceberg. We still have data, which is still been written, with handwritten notes in scoreboards and all these, which are still lying in different offices. So we still have a significant work to do in terms of consolidation, which effectively would be that.
Yes, we have made up the data model. We have consolidated the right amount of data, which was digitized available, but then we still have some of the consolidation to do, conversion of handwritten notes of scoreboards and all of these. So the first part of it, just to go back to your question, is to make sure that we identify the sources, identify and understand how our roadmap is going to be for the data modeling. Structure the data model, and also have provision over migrating all of these data sources as we move along.
It must be nice because it's something that intuitively maps to events, right? Sporting events, scores, that naturally maps to an event model?
Exactly. Because you start off when you come into any sport and you have the players, and you have your teams. And then the format of the sport is defined by the events. And if you consider given an example for 10 people who have followed tennis, you have different tiers of tennis events, where you have Master 500, Masters 1000, or grand slams. And that is exactly what we wanted to conceptualize from WTT perspective as well. We have different tiers of events starting off with the youth events with under 19 17, 15, 7, 11 players promoting the sport, the younger generation.
And then the second part is the senior under professional sports where we have equivalent to Grand Slams. We have the Grand Smashes, we have Star Contender, Contender at different levels of the events. And this is where exactly when you start off with the event structure, then you have all of the match related information. And then the stats, whatever happens in the match consolidating over different analytical points that comes out of the stats.
Yeah. So you are providing the service, not just for these big headline events, but also for more local matches?
Exactly. I think, our vision when it comes to our five year roadmap is to even a lot of our products to be used at the club level. Even people playing in a table in their club or whatever, they should be able to play, record the matches and then be rated. And effectively can be moved over, and if they really want to move ahead, it's always fun to be rated. I play with you, and then whether I'm better than somebody else, and then I'm rated against another friends group. And so, that is the intention. And that our ecosystem includes all of the fans reaching out to the last fan base.
Yeah. It's funny, because my son plays football and glimpse in the modern world, he's on his phone checking live sports results of the rest of his little, 12 year old league. And there is predictive models, so he knows what the score should be for the match he's playing tonight. It's the modern world, right?
Exactly. And that is the reason that we are looking at the ranking. So when you play a mature games or even at the professional level, you always want to be marked against each other. That is where you are individual ratings and ranking come in. All of that, it's not just about ratings. We are talking about now recording your scores. It's not just playing your scores as playing your matches, but at the same time your match scores are recorded. So that is the level that we want to go, it's not just for professional sports, not just for WTT all of our products that we are trying to build. It's not for internal consumption. Primarily, yes, to start with WTT and ITTF. But our intention is for reaching out to all of our member associations to provide these products, to the clubs or to individual players who could play in their backyard or having a table or in their Raha, in their club and start to be involved in table tennis.
Yeah. Because this is often the thing I sense in sports, sure, there is a big commercial arm to get these big headline matches, but there is also the push to just make things richer for everyday players, to engage the world in playing the sport?
Exactly. And you touched on the topic where you wanted to say the predictive analysis.
And this is exactly what we want to get through. This is not just for the professional sports. Effectively, these days they start to have these year end results. When you are rated, you want to see whether I could be the world number one on a specific discipline, whether I'm a men singles player or a women's singles player, what do I get out? What do I need to do to be the world number one? And these are the things that is a predictive analysis comes in. The same level when you are looking into fans, this is also the similar situation. Should I work with my friend or do I compete with my friend, or am I the best in my group? That is the level that we wanted to get to because-
It's not a match going to be easy or I have got a chance to bump my ranking to beat this?
Exactly. It's a comparative analysis. So that is exactly why table tennis as a sport, we've not even touched on yet, our roadmap still goes into e-Commerce, merchandising, gaming. These are things we still have in the roadmap. Which again, AI data, you talk about only sports data so far. We also have AI data, which effectively means when I'm playing a shot, the speed of my delivery, the spin rate at which it is being delivered.
Oh, you are recording things like spin rate?
Yeah, we are. So which is effectively, we have started off on that with engaging with our partners. And we started off the last two events back in March. We have introduced that. We are consolidating that information as well. And all that helps with Confluent, because obviously every AI data is also a betting data point for us. It gives a player at the senior level, if I'm going to be winning a point at a specific speed of delivery or a spin rate that again, consolidates into a data point and that does a revenue point as well for Confluent.
Yeah, okay. So this takes us naturally. And let's talk about the real time, how data is dealt with as it's being played. Tell me about the real time nature of your setting?
So the way that we have designed our architecture, as you had promoted Kafka, it's our central nervous system obviously, it's the communication of entire system. Even before we get into the matches, we have used Kafka to integrate the different applications and systems. Right from the time that a new player, for instance, Kris, you are entering it as a pro into table tennis-
Seems unlikely but let's pretend.
Let's start it off. And then you have your data entered as a player. And once you have an event, let's say coming up in June, there is an event in Singapore, that you are going to be registered with. You are setting up the event infrastructure, even set up all of these even tiers, information about where the venue will be, how many players, you also have even structures where you talk about knockouts, you talk about qualification round, so you can continue to configure the event. And then the next part is you've got to have your player entries or event entries. Who is going to be participating in it?
And that automatically comes through Kafka. So the information of Kris, moves over with the information on event. And Kris is going to play this event in June. And then if there is cases where the structure has to be, now, it's going to be, let's say in one of the Singapore, indoor stadium is going to host. We would have our resource system established there and set up. And we are using a touch pad for the umpire. So the umpires start to do the real time scoring. It's not the manual scoreboards anymore.
So just to give an insight about how the metadata flows, every time the umpire scores in a table, in a table tennis event, the data moves from the venue anywhere in the world to Confluent Cloud, and then gets sent out to individual consumers, including our mobile app. And that we have launched back in last year, 2021. As well as websites, betting organizations, we provide live data feeds to streaming partners, to broadcast partners, all are real time. And that requires these SLA, which is exactly why we have relied on Confluent, and especially considering that the challenges that we have with managed cloud platform, it helped us to fast track our development.
Yeah. This is a thing we often talk about in real time, because we are talking about soft real time systems in which a late event is valuable, but how valuable, right? So this particularly at the edge with sports, a finding out when the score happened is very real time. Not quite AI cars, but you need to know if that point was scored now, because it's going to book makers and televisions.
Exactly. And if you talk about real time streaming, as you just inform, where you have your broadcast and you have to stream it with the live score and the data has to be connected. Not just that we also use, even for media archiving, the same triggers effectively using Confluent. So give you an example, when a match starts, you want to have to start the video archive. So effectively you start off with the trigger, that is an event for us which is even managed through Confluent.
So you saying that the umpire is kicking off the recording of the live feed?
So you have your match as a [inaudible 00:21:23]. We Have structured based on Olympics data structures in terms of the events. So we capture a match start and then the video streaming starts automatically. And we are using Azure's media services as well for live streaming at this stage. But then every part of, let's say you want to, the match is completed then you have a price money distribution, all that is also triggered through our entire umpire's touch pad application, which is connected through cloud-native Kafka.
So all their rankings and their prize money distributed in real time.
Exactly. So effectively when you have your event, obviously there are rankings. In fact, the way that the infrastructure and the architecture that we have put in place with the help of Confluent Kafka, as well as Microsoft hosting services, it has allowed us to publish ranking on a weekly basis. Previously, it was a monthly and now on every Tuesday morning, 8:00 Singapore, we are able to publish the ranking. Reason, anywhere in the world that the events can happen, we are able to consolidate the events real time with the help of the data consolidated through cloud-native Kafka. And then we are able to process the results, and then calculate the rating and publish it back to the website.
That's a huge improvement from monthly. But you know I'm going to ask this question, why is it not up to the minute?
The reason is the events run because it includes to the end of the event, meaning of the finals.
Oh, you have to wait for-
You can still do. You can do progress rating, there is nothing to stop them. But as a regulation for ranking at this stage, we complete the event and then calculate.
Right. So the problem is the real world is too slow?
Yeah. But there is also cases where there could be player penalties for instance. So those are things that will only be decided at the end of an event, which you'll have to wait until the event is complete. If there is any penalties or if there is any other changes that will happen, people do in table tennis, you have cases where you have walkovers and other things. So there will be considerations only done near the end of the event.
But that manual consolidation at the end is gone, you are saying?
It would still be triggered through the process where we will get the results, but we have to wait until the finals. But as you said, there is nothing to stop in terms of getting real time rating because we get the results for every match.
I'm not connected to the real time thing. Call me naive, but I hadn't realized book makers are demanding data SLAs of you these days?
Exactly. Give an example from a betting side, you are talking about, in table tennis we have a game point 10, let's say, every game. And if I have to decide who will be the winner and an average table tennis match is about 35 minutes. And you need to have this data, which is 10, 0, to 10, 11, 9 has to be within the milliseconds obviously, from the time that the score is down in the venue. So which is exactly why the SLAs are very critical.
And this is one of the reason that our Confluent helped us, with very limited effort in terms of setup and configuration. Manage cloud was perfect for us with regard to having that availability, the meeting, the SLA. And thanks to Confluent, we are able to meet those SLAs so far. And we have the reliability of the platform, which is pretty much what we need with a limited effort in terms of maintenance. So pretty much a best fit for us for Confluent. And thankfully we have an expansion plan with regard to reaching out to all of the fan base and the member associations. And we are in for a expansion which will likely be in three years time, we would see that reach out to the fan base.
Okay. So give me a sense of your roadmap for the future. What is it you are trying to explore?
I think, as I started off, first thing first, we need to start moving in with all of these AI. We are even looking at blockchain NFTs. And the next part is gaming. We've also been looking at gaming. And also working towards member associations as I started, we wanted to license the products. We will want to make sure that all of the systems that we have built, including the data models, the structures, it's the same structure that the member's Association is going to use. The table tennis as such, it's the same structure. So they could be using our resource system. They could use the same central cloud-native Kafka stream, and they still be able to produce the stats. And for us, we will be able to manage all of the data as a central single source of truth, which also helps us expand our fan base as well. Every data point that we are going to get out of the license product, can be consolidated.
And so that's the other. And then, we are also looking at moving towards real time stats, because we wanted to go stats at the level that we have now, table tennis is a very standard set of stats, data points. Now, we want to go deeper into, with all of these AI integrations, we want to show real time stats, which is in the venue building onto the sports' presentation staff in terms of broadcast to streaming. So a lot more in place in the roadmap in the next three years. And obviously, looking forward to meeting all of the strategy and vision of the form.
Oh, cool. So this is stuff like you see people commentating on tennis matches for instance, and he's hit the ball 103 miles an hour, and that's the fastest he's hit it for the past two seasons?
Exactly. I think you got it. I think when you see that, and it starts to get into those stats, you think about, "We have Jiayang Dong, is the world number one player or anybody else with the top 10." You could see that this is the fastest to serve unlike the top 10 player. That's the level of stats, or it could be the thousandth point for that particular player during a month. So these are all stats that get more fans engaged. There is other aspects of that as well is our... When we said about the roadmap is also getting to know about the fans better, the demographics better in terms of real time clickstream. Obviously, we are using website, we are using the app. So the first party data coming out of it and also third party data, just to make sure that we are building into the analytics platform to make sure we can reach out to fans better, and then OTT, e-commerce and other products also to be integrated into the system.
Right. Yeah. So as you are sending them real time data about the matches, you are getting realtime data about how interested they are?
How interested they are, yeah. So it's just the full, not just the sports' data and then getting to know the fans better, we'll also be one of the things that we were also looking into.
So you can serve them better and that feeds back into the non-commercial arms.
Exactly. So this is a full ecosystem that we wanted to build, not just for running the events or running the matches. It's just about building that full fan engagement. Through spreading the sport to every part of the world. Because again, as I mentioned, it's a global sport and we have 226 member associations right through. And we have a lot of potential to reach out to any fan in any part of the globe.
Yeah, absolutely. So you've been going, what 18 months if I got this right?
Tell me about your next big match and how WTT is involved?
We do have this year, I think, as we are all struggling with pandemic and we are coming out of the pandemic situation. We had some ups and downs in terms of events being globalized. And just to give some insight, about 60% of our fan base is in China. And we have not started on running the events in China at this stage.
Table tennis is huge in China, right? And they are not broken into that market yet?
Because of the pandemic situation where China is partly closed and we have challenges over hosting events there. But now we are going to be having events back in Q3, Q4. We have plans for starting off the events in China, and that will be a huge market for us. And effectively leading up to it, as we said, we had our events in Europe, we had events in East Asia, and we had in Singapore, and we are now moving over to a bigger market, which is China. And once we have that China, then it becomes in the number of events, the volume of fans that are going to come in, it's just going to be awesome.
You are absolutely poised to go global.
Any scaling worries?
With Confluent, I'm least worried because... And one of the things that also helped that I have to make a note here. Right from the start, my journey over digital transformation. And the moment that I identified Confluent Kafka to be the main product for me on the data messaging side, the professional services coming out from the Confluent team were always critical. They were part of the journey in terms of helping me out in defining the architecture, trying to run through the scalability, availability, security considerations, compliance cases. This was too detailed when it starts to get the support from Confluent. And this was right the way through, even quite recently we had architecture review session with Confluent team as well.
Okay. That's cool.
Just validating the architecture. And then they have been really helpful in helping us out, not just on architecture, but also helping us to meet our strategic vision and meet our roadmap and priorities.
That's cool. Okay. So I shall have to go and speak to those guys and thank you.
They have been really helpful. And then the accounts team here in Singapore is also been really helpful in terms of, having the right set of services and the right pricing tiers, that again, you don't have to. Any cloud services these days, you do have the option of where, you can use incorrect pricing tier, which is not fit for you. Obviously, they guided me through in terms of having the right structure that can accommodate and be used for my purpose.
That's very nice to hear. But that's a nice commercial note to end on, but I think we should get back to [inaudible 00:32:35]. There is a big table tennis table out in the lobby. I think we should wrap up the podcast and go and have a game.
Excellent. Yep, we will do. Thanks for your time, cool.
Thanks for being on Streaming Audio.
Thanks, Kris. Thanks to you too. Bye.
And there we go, a window into a world that's very different, but still full of familiar challenges. I should report that despite all I learned from that, Sam, I am still useless at table tennis and I will be sticking to my day job. What is my day job you ask? Well, part of it is reminding you that if you want to learn more about Kafka, realtime data and building event systems, Confluent Developer is here to help. If you head to developer.confluent.io, you'll find everything there from beginners guides to Kafka, to high level architectural patterns, deep dives, and some inspiring blog posts. It's all free. And it's written by some of the best people in the business. So take a look.
And when you are ready to run Kafka in production, head to confluent.cloud, you can get a cluster up and running in minutes. Add the code podcast 100 to year account, and we'll give you $100 of extra free credit to run with. If you've enjoyed this episode, please do take a moment to like, subscribe, rate, review, click the notification bell, all those good things, or just drop me a line on Twitter, my handle is in the show notes. And we always love to hear from you. And with that, it just remains for me to thank Vatsan, for joining us and you for listening. I've been your host, Kris Jenkins, and I will catch you next time.
Reimagining a data architecture to provide real-time data flow for sporting events can be complicated, especially for organizations with as much data as World Table Tennis (WTT). Vatsan Rama (Director of IT, ITTF Group) shares why real-time data is essential in the sporting world and how his team reengineered their data system in 18 months, moving from a solely on-premises infrastructure to a cloud-native data system that uses Confluent Cloud with Apache Kafka® as its central nervous system.
World Table Tennis is a business created by the International Table Tennis Federation (ITTF) to manage the official professional Table Tennis series of events and its commercial rights. World Table Tennis is also leading the sport digital transformation and commercializes its software application for real-time event scoring worldwide. Previously, ITTF scoring was processed manually with a desktop-based, on-venue results system (OVR) —an on-premises solution to process match data that calculated rankings and records, then sent event information to other systems, such as scoreboards.
To provide match status in real-time, which makes the sport more engaging for fans and adds a competitive edge for players, Vatsan reengineered their OVR system to allow instant data sync between on-premises competition systems with the Cloud.
The redesign started by establishing an event-driven architecture with Kafka that consolidates all legacy data sources, including records in Excel along with some handwritten forms (some dating back 90 years, even including records from the 1930 World Championship).
To reduce operational overhead and maintenance, the team decided to stream data through fully managed Kafka as a service on Azure, for a scalable, distributed infrastructure. Vatsan shares that multiple table tennis events can run in parallel globally, and every time an umpire marks scores in a table, the data moves from the venue into Confluent Cloud, and then the score and rankings are sent to betting organizations and individuals on their mobile apps.
If there's something you want to know about Apache Kafka, Confluent or event streaming, please send us an email with your question and we'll hope to answer it on the next episode of Ask Confluent.Email Us