Build to scale: queueing and latency on Twilio
Delivering messages at scale is a top-of-mind consideration when building applications for high-traffic events, such as Black Friday sales in the U.S.. Therefore, we've created a guide of best practices that you can follow to ensure your applications with Twilio scale easily.
Imagine you have a list of 1,000 products that go on sale at 11:00 AM the morning of Black Friday. Now, you have one million eager end users who want to be notified of your Black Friday deals as soon as they're live. You're planning to send out a blast of promotional SMS messages to that entire group at 10:00 AM Black Friday morning to let them know the deals will be live in an hour.
You also know the best products run out within a few minutes. To protect against automated bots and to ensure that your potential audience has an equitable experience, you need to incorporate an OTP code verification step as part of the checkout flow. In this case, you require two messages to safely and securely facilitate the sale.
Based on your use case, on Black Friday, you have two separate use cases for Twilio Messaging: one for promotional messages sent an hour before the sales go live and one for OTP verification messages during checkout. For each of these use cases, we recommend creating a Messaging Service with a dedicated Sender Pool and Validity Period setting.
- Message 1, Promotional - The end user gets a marketing link in a message just before your Black Friday sales go live. The link takes them to an application where they have to login or signup.
- Message 2, OTP - Upon signup, the end user must enter a one-time-passcode (OTP) pin for logging in. This message has to arrive immediately, so that the user can purchase before the items run out! For this example, let's conservatively assume that 2% of your audience clicks the link and eventually, 0.1% buy.
You can think of a Twilio Messaging Service as a bundling of both senders (in this case, phone numbers) and settings for a specific messaging use case. When you need to consider messaging at scale, a Messaging Service handles intelligent routing and features like opt-out management for you, across multiple senders. So instead of configuring multiple individual numbers, you can create a Messaging Service and add as many senders to the sender pool as you need. The configuration will apply to all of the numbers in your sender pool.
Info
Whenever you are using multiple Senders to service a single use case (as in this example), you should be aware that this practice is governed by various local regulations, depending on the country into which you are messaging. Twilio strongly encourages you to be aware of and adhere to these guidelines. On this page you can find current guidelines for sms use in many countries outside the U.S. Among U.S. carriers, the practice of using multiple senders for the same message as a way to circumvent regulations is known as "snowshoeing"; here's is Twilio's policy regarding this. Here you can find guidelines and registration steps for the use of Toll Free numbers for sms messaging within the U.S. Finally, the use of 10DLC numbers to send sms messages within the U.S. is governed by new regulations you should definitely be aware of before choosing a scaling solution with large numbers of 10DLC numbers in the U.S.; please see this overview document.
When you send a Message with a Messaging Service, the Service queues the API Request before sending it to the specific Sender, then onto the Carrier for eventual delivery to the end user. Instead of a from phone number, you will enter a Messaging Service SID in your API request to Twilio.
In a previous step, you calculated your MPS requirements and added a buffer to make sure your messages were sent in the appropriate window of time for your two use cases. To meet these requirements, you'll want to add different Senders to your Messaging Services' Sender Pools.
To adhere to carrier requirements, each Sender (phone number) on Twilio has specific MPS limits. These limits vary by both Sender type and location. For example, Short Codes have a 100 MPS limit. A Long Code in the US has 1 MPS, but a Long Code in the UK could have 10 MPS.
Now that you have Messaging Services with the appropriate configurations and Sender Pool, you can set up your messages to be delivered on time with the Twilio Programmable Messaging API.
- Promotional: Queue them up in the morning so your messages have plenty of time to get delivered.
- OTP: Set up your application to automatically send OTP messages as needed.
Warning
Latency is calculated as the time on Twilio Platform (the time taken by Twilio from receiving an API request to sending the Message to carriers). It does not represent any downstream carrier-related latency because Twilio does not have full visibility into handset delivery.
You can keep an eye on your messaging application using Twilio's Messaging Insights in the Twilio Console. (Check our our doc on Exploring Messaging Insights.)