In today’s episode, we talked to Shanif Dhanani, co-founder & CEO of Apteo. He shares with us his backstory, how AI predictions and Apteo works, machine and deep learning, and what's next for Shanif and Apteo.
Apteo helps ecommerce brands personalize their marketing campaigns by predicting what their customers will buy next. Prior to Apteo, Shanif was the lead engineer and head of analytics at TapCommerce, an NYC-based ad-tech startup that was acquired by Twitter.
His Backstory
Shanif Dhanani: First of all, thanks for having me, super excited to chat with you. It's rare I get to chat with folks from the other side of the world, so looking forward to it.
So, you know, really quickly about me. I've always loved technology you know, for me being able to sit down and program something has always been sort of an interesting exercise. So before, the way I got into it was I just kind of taught myself how to program in JavaScript and HTML, and then taught myself along the way. Did that in school. And ultimately, you know, I try to be well-rounded as well.
So I ended up going to business school. When I was there, I ended up joining up with a couple of my buddies and we helped create this company called tap commerce. That was kind of my first real startup. It was an ad tech platform, when mobile phones were still getting paid, that was doing engineering for them and sort of got back into statistics and AI, which I had done in school. So I did that.
We're very fortunate. We got sold to Twitter. So I worked at Twitter. You know, as you saw for a few years, I was doing all sorts of interesting stuff. Everything from predicting, who's going to click on that ad, all the way to like understanding and optimizing their onboarding flow to see if we could make it easier for people to sign up for the app then decided to start my current company, Apteo, which again, we're using data and AI to help e-commerce brands.
So hopefully that was not boring or snooze and dose inducing, but that's kind of my background.
Connor: No, no, no, no, no. That's awesome. It's nice to hear the full story.
I didn't know that you were like bought out and then you were then working at Twitter.
Shanif Dhanani: Yeah. And we were very lucky, it was one of those days where everything was just starting, starting from scratch, with the world of mobile and mobile advertising. And so we were just kind of at the right place at the right time.
How AI Predictions and Apteo Work
Shanif Dhanani: It's funny because I think I've mentioned this before, but today when you hear the word, AI, people tend to think of like robots or Terminator and stuff like that. But in reality, it's actually just math, it's math and statistics. And so you can pretty much quantify everything today. How much, you know, is somebody male or female, or how many times did they buy a product or even what zip code, you know, in the US what zip code are you living in?
And so what's really interesting is there are certain pieces of data that if you collect it for lots and lots of people, you find these, these patterns that emerge over time. So if you're a brand, if you're an e-commerce brand and you've got good customer data, you can actually take everything, you know, about all your customers and all your products, you know, everything you're doing, actually use those trends and patterns to incentivize people to make new purchases.
So for example, you know, a lot of the brands that we work with, or let's say fashion companies, some in some of these companies have, you know, tens of thousands of products, but you can group these products into similar categories like dresses and shoes. You can actually predict which customer is likely to buy from which category like a dresser issue.
And you can even predict it down to the, to the product level just by analyzing what you know about that customer, comparing it to other customers, seeing what those customers have done in the past and what they went on to do in the future. So I'm not going to get super technical, but it just comes down to using all the data. You have to take a look at what you know about somebody, historically what did they go on to do in the future? And then applying that to what, you know today.
Connor: Just quick though. So when the predictions, like how accurate, I mean like generally speaking, how accurate can you be when you look, it's like a lookalike audience kind of thing, right?
Shanif Dhanani: Yeah. There's a few ways you can do things. So there's a few ways that we use predictions. One of the ways we do it as we try to figure out which products, for example, people are likely to buy. And then we group those, those people together.
So if you're a really large store, so we build, we build these AI models individually for each store. So if you are a really large store and you've got a bunch of data, you can get pretty accurate. Now you can have, there's certain metrics that you could take a look at in the world of AI there's area under the curve, there is a RFC.
What we like to look at though and these are some of the things we look at, we look at when it comes to accurate predictions, what's the recall, which is different than the accuracy. Like how good is the model that actually figuring out if somebody goes to buy product next different stores will have different metrics for that. And so if you're thinking about this at an aggregate level, it's hard to compare a brand. That's got maybe 5,000 with historical customers with somebody that's got 2 million.
So when we think about accuracy, what we actually try to think about is how is the marketer using these predictions in their business? And so what we look at is going one step further and saying, okay, regardless of how accurate the model. Oh, the AI is if we are grouping people together based on the products that they're likely to buy, are they actually buying those products? You know, if we're sending them an email for a particular product, are they actually buying them? Or if we're saying, look, this group of people are likely to be your biggest spenders in the future. Are they actually going on and spending more money in the future?
And so from that perspective, we do pretty well. So we are usually able to drive a lot of incremental of specific products when we show those products to the people, let's say, for example, in an email or an ad, but you know, people, people are funny. They're not going to make a purchase. Maybe the first time you send an email or the second time there's this kind of art and science to marketing where the science is. Okay. Who's likely to make a purchase, but you have to be able to kind of incentivize them. So do you send them discounts? Do you target them on Facebook and Klaviyo and SMS, or do you just use one channel?
And so I've been going into sort of the application of our AI, and maybe it's a little bit beyond the scope of your question, but when we're thinking about the AI, it's really, how do we incentivize people to make purchases? And from that perspective, especially for larger stores, we do very well from smaller stores. We can actually still drive incremental purchases. Sometimes $2,003,000 a week for a really small store.
So that's kind of how we think about it, but if you want, you know, I'm happy to talk exactly how the AI works, which is you try to predict the probability that somebody is going to make a purchase on each individual product. And then you compare that to the likelihood of them actually buying it. Like, did they, did they not buy it?
Connor: Yeah. So that's the feedback loop, pay that final piece. Did they or did they not buy it, you then put that back into Apteo?
Shanif Dhanani: That's exactly right. So every 30 days we basically updates, you know, an AI system is the same as like a model. That's what I'm going to refer to it as a model is basically. This mathematical equation, this monstrous equation. If you can think about it, that takes everything, you know, about a person and spits out a probability.
And so every 30 days when we update this model, you know, we add in all the data that we've gotten over the past 30 days, and we say, okay, did this person go on to buy something? Did this person not go on to buy something? And you close the feedback loop, but you continuously do that. You refresh it every, in our case, every 30 days.
And then you actually use that updated model to make an updated prediction once a day. So you update it with new data. Every 30 days, you come up with a new prediction once a day, because time as time goes on, your predictions will change. And so updating your predictions once a day will take that into account.
Now this isn't necessarily how most, how every company does it at Twitter. We had a system that could actually take into account every time you saw an ad, that was a feedback that did you, or did you not click it? And other systems, maybe they don't even update, you know, at all, maybe they're just a static system that's used for one small thing, and then they forget about it.
And so there's different ways of doing it. But yeah, to answer your question, actually, the most, one of the most important parts of the AI is saying to AI or giving the AI data about, did somebody actually go on to make a purchase or not? Because that's, that's actually how it learns actually.
How Shanif came up with Apteo
Shanif Dhanani: I've always loved the idea that you could take data, you can take something that, you know, and predict the future. So I've always been sort of enthralled by data science. Now you combine that with one of my hobbies from about 10, 11 years ago, which was options trading.
And you've got this sort of beginnings of a system where somebody like me was trading stock options to make some money, doing it with my gut instinct, wanting to make it a little bit more systemic. And had this interest of using AI to basically help me with my options.
So actually when we started Apteo a few years ago, we were applying AI to finance. We were actually using it to try to pick stocks and try to figure out which stocks were going to do really well. And we actually built our first products around that. We had this sort of stock picking system that anybody could sign up for. I think it was really simple. It was like a list basically from one to a hundred, or we tried to sell that thing and nobody, nobody would. This was about four years ago.
It's funny, you know, it's funny because we tried to sell to that individual retail trader, like you and me, and nobody really cared because they weren't managing, nobody was really managing their money and they're all just putting it into index funds and letting it sit, except for like a few people.
And you would try to sell it to finance firms and, you know, they would have required these lawns sort of back tests and they were very skeptical.
And so we had to pivot, we had to sort of change what we were doing and we eventually changed. We found that we, we built sort of a bigger system where it was analyzing data, not just from finance companies, but from any company you could put your data into the system. And it would spit out predictions. And we saw that the only people that were using that system were marketing experts from e-commerce.
So we said, all right, screw it. Let's just go hardcore into the world of e-commerce. Let's talk to a lot of e-commerce marketers. Let's figure out what they want. And so we eventually built the system for them. And what's funny is two years after we pivoted. People kept coming up to me. They're like, Hey, can you rebuild that system? Like your stock picks that amazing. Like, I can't do this myself. Can you guys rebuild it? Can I have access to it? I'm like, guys, you should have told me two years ago.
So yeah. But that's how we got to where we are today. So lots of talking to people and experimenting and sort of that quintessential product market fit journey that most startups go to.
Challenges they encountered along the way
Shanif Dhanani: Oh, that's interesting. Startups are tough because you basically tried to build something that you're not sure if it's going to work or not with sometimes people you've never worked with before. You know, I had a couple of co-founders who I was working with. They decided, Hey look, like not really able to make financial, like I'm not able to get the financial needs that I, I need to have I'm working for a startup. So they have to go and take full-time jobs. So that was tough.
You know, anytime somebody has to leave because they just, you know, they can't work, they can't afford to work at a startup. That's always a little bit tough. There have been days where, you know, a couple of invest like three or four investors who I thought, you know, might come through all sorts of. No, I'm the same day. I was tough. So there's definitely like, yeah, there's definitely lots of those, but at the same time when you close a deal or when you, when you talk to a customer and they're like, hey, you just made me, you know, $20,000, those are days that pick you up.
And so for me, I really liked just helping people. I've sort of had to work really, really hard to make my way through life. And so if e-commerce owner are very similar and that they're working really, really hard on their sites and their stores to gain financial freedom. And so if I can help somebody like that, it actually makes my day. So yeah, there's good and bad and those are some of the, yeah, those are some of the examples that I can come up with right off the top of my head.
Connor: True. Has it been the thinking about how it could have gone differently? You've been doing this, you've been out on your own with your co-founders for the last four years. How has that impacted your wider life? Like if you move houses?
Shanif Dhanani: Yeah. I'll kind of phrase it in two ways. One, I was very fortunate to have the outcome that I had from tap commerce, which was the first startup I did. So that gave me a lot of financial freedom to experiment. But when I was doing tap commerce, I was sleeping on a friend's couch for six months because I couldn't afford rent. And I was trying to figure out, you know, how was I going to pay for food? And how's it gonna pay it back? That was perhaps one of the most stressful days of my stressful times of my life.
Now, fortunately we got through it. I had some brilliant, brilliant co-founders. I feel very lucky to kind of have been along for the ride with them. Apteo has been a little bit different as a founder myself this time, the first couple of years, I was very, you know, the identity of the startup was very closely tied to my identity. Like if something wasn't going wrong, it means I wasn't doing a good enough. I was always of the opinion that you got to got to work, work, work.
If you work enough, it's gonna, it's going to work out. And then after, you know, a couple of years of doing that and sort of not much change, you sort of realize that that's not going to be the case. Like no matter how hard you work, sometimes it might just be something that's not going to work and if something goes wrong, it's maybe not always your fault.
And so the way it's kind of changed my life is it has. Sort of helped me see perspective, like a little bit more when I was a little bit younger, I was very much on the opinion that, you know, if you just work hard, you're going to be able to do really well in life. And so it just worked really hard and you're going to be okay now I'm kind of like, there's the journey, which matters a lot, a lot as well.
So like be good to people, enjoy what you're doing. Try to figure out new ways to come up with, you know, a solution to a problem. Those are all valuable as well, regardless of, you know, the.
And if I'm not killing myself to do these things and I'm going to be in good shape. So that's one way sort of getting perspective on, you know, the journey and the work ethic, I think is one thing that's affected me. The other thing, you know, that you might find surprising is at one point, I just, I thought we were going to have to shut down and I had to come to terms with that. And by coming to terms with it, it actually was like a huge.
And so the other thing that I've kind of learned, fortunately, we haven't had to shut down yet. We're just hopefully going strong now. But the thing I had to learn when I have to come to terms with it is I look at something you tried and if it doesn't work out it's okay. So, you know, the sort of emotional release that you get from something like coming to terms with something not working out, was it's a really good sort of experience and a learning experience to have.
Connor: Oh, thank you for being so vulnerable. Yeah. That's awesome.
Shanif Dhanani: It has been by far the hardest thing I've ever done, but also I'm the type of person. I find it a lot more sort of valuable. I find it a lot more inspiring to work on something where I can make a big impact rather than maybe working on a corporate job where I'm not making a huge impact and not making it.
You know, a big move. So for me, I like it. I don't know if I could ever work at another corporate job. I also don't know if I should ever do another startup after this one, but I'm not thinking about it quite yet. I'm still working hard to finish off this one, so we'll see how it goes.
Biases in AI
Shanif Dhanani: Yeah. So let's talk about, you know, machine learning and deep learning are such a big topic. I'll talk about what you just said first, which is the bias in terms of introducing bias into the system and how can you start to avoid it? So if you go back to what I said a few minutes ago, AI is at its core. It's just math and math is basically just taking a bunch of data and finding the relationships with them.
So when you are basically let's say training a system to detect faces, screen candidates job opportunities based on their resume. You're taking a whole bunch of stuff that you already have in terms of the data images or resumes or whatever it is, and you are linking them to the outcome or to whatever it is that you want the system to learn.
Here's a good example. Maybe you're taking resumes of your current employees and saying, these are really good employees. These are really good resumes so that whenever somebody who's not a current employee submits the resume, they find something that looks like your current employer. It's going to send them to the next stage.
Now, what happens if your employee base is not diverse at all? What happens if they're mostly male or mostly white or mostly engineers are mostly from one part of the world what's gonna happen is the AI system, because it's just, math is going to find people who have very similar resumes. And that means it's going to discard discard people who don't have similar resumes, but who might still be a good fit for your company.
So when people are talking about bias, when it comes to AI system, When it comes to this sort of by it's funny, because bias is actually a term in the world of data science that has a specific technical meaning, but most people are talking about human bias, which is where the AI system it's not doing what you think it should be doing, but it's doing exactly what you told it to do, which is not always what you want.
So in order to set up a system like that properly, there's a whole bunch of things that need to. Like first you have to understand is bias. Is human bias, something that I have to take into account for the project that I'm working on. So for example, it doesn't matter if our system, if the Apteo system, maybe miss qualifies somebody as likely, or not likely to buy a product, maybe not, but it doesn't matter if an AI system flags you for screening at an airport because you look like you might have, I don't know, something on you or whatever.
And then it matters, then it's a real big problem. So once you identified that it's a problem, a lot of work has to go into preparing a data set that can account for any biases that you, as a human can come up with. What's kind of weird here is that we, as humans need to input our judgment into what could be biasing, a system that hasn't even been built yet.
So we obviously know things about like race, gender, socioeconomic status. We can account for all of those biases by putting together a data set that. You know, equal examples from, from everything we care about, but what happens if you, if you miss something or if you forget about something that you didn't know was a bias and then you train an AI system and I go through and encounters that bias.
Well, then it's going to be biased. And so you're going to have to figure out how bad is it again? How bad is it if an AI system is bias or not. And that's why I still think there's more value in what I call like man and machine or human and machine rather than just machine itself. I think an AI system can inform decisions and sometimes, maybe it makes it 99% of the decision, but sometimes you might want a human in the loop to figure out if that decision should be carried forward or not.
And so that's why I like systems where you take a lot of the app and analytics and the analysis out of the work, you can give that to the machine because the machine is really good at that. And then you take what the machine spits out and you say, okay, let's move forward or let's make some tweaks and go from there.
That's sort of the bias that you hear about a lot from the media perspective. Does that sort of answer the question in terms of what you're looking at? Anything else you wanted to dive into there?
Connor: Yeah. That's a really succinct explanation actually. How many people do you have in that room? Like do you have to be a pretty smart person and be like, okay, I'm going to decide.
Shanif Dhanani: It's funny because you almost want, I don't know if I like this term or not, but you always want a multidisciplinary team where you've got sociologists do that. You've got people from multiple walks of life. The worst thing you could do is just have a bunch of AI engineers in a room trying to figure this stuff.
So if you're Google and you're trying to build a system like an image classification system, or, or even like, if you're somebody building a facial recognition system, you're gonna want a good diverse representation of people in the room to figure out, you know, one, should you even build this system?
And sometimes you don't have an option. Sometimes you have a contract or sometimes that's what your company does, but it's worth talking about, should you even build it assuming you have to build it. How do you build it? So that one, it does its job, but two, it doesn't sort of bias you. So I think like you're just going to need enough people to talk about how to do these things.
And I think you're going to need a system of checks and balances where just like with any organization, you've got the people who build it. You've got the managers, you've got the people who are the stakeholders. You don't want to make sure you've got a similar setup. So that you minimize the risk of something falling through the cracks.
We, as humans are never going to get everything right, but we can minimize risk. And that's something that you could do with an organizational team like this. So it's funny, you you've kind of heard me talk a lot about the societal impact, the human impact and not so much about the tech, because the tech is straightforward relatively straight forward.
Like let's say, you know that you have too much of one class. So when you're trying to predict cancer for you. Something like 99.9, nine, 9% of cancer cases are going to show is false. And so when you've got a system that's trying to predict cancer, you can do things on the technical side to make sure that you don't miss a positive case.
You can sort of bias the system, technical bias, the system to overpredict on cases where there is no cancer, where it says yes, there is just the case youdon't miss it. That's all straightforward, but the harder part is the human part and the societal?
Connor: Real quick. So how is it predicting 99% of cases in cancer? Because we all are going to get cancer eventually, or?
Shanif Dhanani: Sorry, maybe I misspoke. 99% of cases are going to say there's no cancer. Like this person here's this person's charts. Their data. They don't have cancer. And the machine is just going to say that by default, because 99% of the time we don't have cancer.
And so most of the cases is that it gets to learn from are, just, are false. And so you have to account for that. There are really good ways to account for that on the technical side, but that's not exactly what we're talking about here. So I wanted to make sure that was clear.
Connor: Are you using these kinds of things at Apteo?
Shanif Dhanani: With Apteo, yes and no. So what we do, we have a couple of different AI models. One predicts how much money somebody will spend in the future, regardless of what they're going to buy. It's just a single number. The other model predicts which products they're likely to buy. And it's a probability, you know, this product in this person, maybe it's zero, 0.01% of this product in this person, maybe it's 84%. So those are two models that we're using.
And from our perspective, There are certain things we have to do. So when you take a look at an e-commerce store, most of the time, you're not making a purchase, like most of the time you've got ton of customers who only buy once or a ton of visitors, most people don't go on to buy a second time or third time.
And so you have to correct for that when you try to figure out which customers are likely to make a repeat purchase, because much like in the cancer case, the model could easily say nobody's going to come back and make a repeat. Because almost everybody I see doesn't, doesn't make a repeat purchase and that's not true, you know, that's not true.
So you have to sort of tweak and adjust the model so that it can make reasonable predictions, you know, knowing this, there is a few things that can help here, but as a data scientist, a lot of your job is doing things like cleaning up the data, looking at the model's performance, tweaking it, trying to figure out how to make it better.
So we are doing a lot of that. We're not doing a lot of sort of human bias correction that like we're not putting necessarily racial information into the models. We're not sort of doing this sort of stuff that I just mentioned. And a lot of that just goes back to one, we're a small startup. And so we just have to make sure we can stick around for the next six months and to, you know, let's say we get a prediction, right? It's a very, very minimal impact to society and business.
And so it's one of those things where it doesn't necessarily need to be done today, but if we get bigger and let's say, we started to, let's say we started to get into other businesses, or we started to like, I don't know, predict stuff like which health plan somebody should be using or something that's sort of out of the world of what we're doing right now. That is something that we would have to think about. And that is something where we would bring on sort of a good team to start figuring out how do we build something like.
What's next for Shanif and Apteo
Shanif Dhanani: You know, with me, the startup life despite my ability to sort of not necessarily tie my entire identity to it still takes up all my time. And so we are in 2022, we're basically making a big push to grow sales and sort of drive more revenue and if we're able to do that, we're probably gonna try to raise a, you know, a notable sort of round of fundraising later this year or next year, and just grow the team and really make a solid goal.
And if we're not able to do that, then it's, it's all good. You know, we gave it our best shot. I'll probably take six months off and then maybe try my next startup. I'm not really thinking about that quite yet, because I got, you know, I got some time to take an idea to the next level and I'm really, really trying to figure out, you know, the best way to do that.
So for the next, you know, four or five, six months, that's really going to be heading. I'm doing a lot of sales and marketing now. So not as much AI as I used to, but hopefully I can bring on, you know, sales and marketers and get back to what, you know, what my specialty is. But yeah, we'll see if I can, if we can grow the business in the next six months, seven months, I think we're going to be in good shape.
And if not, you know, I still got a bunch of ideas in my head that I want to try out for the future. So, it's hard to say right now, but that's kinda what the plan is.
Connor: Awesome. How are you going to go about growing the business? I mean, apart from the funding side, is there any products you can bring out or are you going to do a big marketing campaign?
Shanif Dhanani: For the past couple of months, it was just me who was doing sales. So we brought on two guys to help do direct outreach, direct sales to brands, and I think that's going to help. We're also doing paid sponsorships and e-commerce newsletters and jumping on as many sort of podcasts as I can. And obviously doing a lot more content. Blogging doing a lot of case studies.
So we've got sort of a push on the sales side and a push on the marketing side. And, you know, hopefully it's going to take off because some of the numbers that we're driving for our customers are pretty, pretty notable. Like we're able to sort of take somebody who's got negative row ads on Facebook and get them back to positive. We're able to take someone's e-commerce account. I'm sorry, email account.
And if I'm making a bunch of money off of it, but if they have good customer data, we're able to start generating like 20 or 30% of the revenue from email. So certainly like the AI and the stats and the math are all really good at identifying strategies. Now we just have to show people, hey, look like this is what we do. Hopefully they're willing to give us a free, like we do a free trial. So hopefully they're willing to try us out and go from there.
