Texting and a unified inbox in ERPNext

This document covers Avunu's open-source messaging app for Frappe and ERPNext: Twilio SMS in both directions, consent handling, group texts, newsletters with merge tags, and a chat-style inbox that puts email and SMS in one place. Use it when you want staff to text customers from ERPNext without a separate tool.

What you get

The app adds these pieces to a Frappe site:

• A Twilio webhook that turns every inbound text into a Communication record.

• Phone number cleanup on Contact: E.164 formatting, de-duplication, and optional carrier lookup.

• An SMS consent flag on each contact, kept in step with STOP and START replies.

• Messaging Group for audiences, and Group Text Message to broadcast to them.

• A Newsletter override that fills in contact fields per recipient.

• A Chat view for Communication that groups email and SMS into conversations.

Source and issues live at github.com/Avunu/messaging.

Install the app

The app declares the Frappe newsletter app as a requirement and depends on the twilio, phonenumbers and pywebpush Python packages. From the bench directory:

bench get-app https://github.com/Avunu/messaging
bench --site <SITE_NAME> install-app messaging
bench build --app messaging
bench migrate

On install and after every migrate, the app registers Chat as a valid view for DocTypes. That is why Communication opens in the chat view by default.

Set up Twilio

You need three things from Twilio: an Account SID, an Auth Token, and a phone number that can send and receive SMS. This document does not cover buying a number or Twilio's registration requirements for business texting; follow Twilio's own documentation for those.

Enter the credentials

1. Open Messaging Settings (a single DocType, editable by System Manager).

2. Fill in Twilio Account SID, Twilio Auth Token and Twilio Phone Number. The token is stored as a Password field.

3. Tick Send via Twilio. With it off, outbound SMS falls back to the core SMS Settings gateway instead.

4. Optionally tick Enable Phone Number Validation (see Normalize and validate numbers).

5. Make sure the country is set in System Settings. The app uses it to read numbers typed without a country code.

Point Twilio at your site

In the Twilio console, set the number's incoming-message webhook to an HTTP POST to this URL:

https://erp.example.com/api/method/messaging.messaging.api.twilio_webhook.sms

The handler is open to guests, because Twilio is not logged in, so it protects itself:

• It rejects anything that is not a POST.

• It throws if no auth token is saved.

• It validates the X-Twilio-Signature header using the Twilio SDK's RequestValidator and your auth token. Failed signatures are rejected.

Twilio signs the exact URL it called. The handler rebuilds that URL from the request, and if a Cf-Visitor header is present it takes the scheme from there. That makes signatures validate when Cloudflare terminates TLS in front of the site. If another proxy sits in front, make sure the scheme your site sees is https, or validation will fail.

On success the handler returns HTTP 204 with no body, so it does not auto-reply.

Receive texts and handle consent

Each valid inbound message creates a Communication with medium SMS, direction Received, status Open, and the subject SMS from <number>. The Twilio MessageSid is stored as the message ID.

If the sender's number matches a Contact Phone row, the communication is attributed to that contact's full name and linked user. Unknown numbers still create a communication, labelled with the raw number.

STOP and START keywords

When the sender is a known contact, the body is compared (case-insensitive, whole message) against Twilio's standard keywords, as described in Twilio's opt-out guide:

The message is still saved as a communication, so you keep a record of the reply.

Matching is by exact phone string. This works because Twilio sends numbers in E.164 and the app stores contact numbers in E.164 too.

Normalize and validate numbers

Every time a Contact is saved, the app cleans it up. (The ERPNext CIS Plus app also normalizes the primary contact's numbers from the Customer form, so existing customer data may already be in E.164.)

1. Email addresses are lowercased and de-duplicated, keeping the first.

2. Each phone number that is not already valid E.164 is parsed with the phonenumbers library and rewritten as E.164 (for example +15555550123). The default country comes from the contact's linked Address, then from System Settings.

3. Numbers that cannot be parsed are left as typed, not deleted.

4. Duplicate numbers are removed, keeping the first.

If you enabled Enable Phone Number Validation and saved both the SID and token, the app also asks Twilio's Lookup API for the line type of any new or changed number. It then:

• sets Is Valid E164 and Validated Carrier Type on the phone row;

• clears Is Primary Mobile on a number whose line type is not mobile;

• picks a primary phone and a primary mobile when missing or invalid, using the first valid number (and first valid mobile).

Lookups happen during save, so a save is a little slower when you add numbers. A failed lookup is logged and marks the number invalid; it does not block the save. Check Twilio's Lookup documentation for availability and billing before you turn this on for a large contact base.

Send texts

Every outbound SMS from Frappe is routed through the app's send_sms hook. With Send via Twilio on, each recipient is sent individually from your Twilio number, so one bad number does not fail the batch. Successful sends are written to SMS Log.

Twilio errors trigger clean-up:

• Error 21610 (recipient unsubscribed): the matching contact's consent_sms is set to 0.

• Error 21211 (invalid number): the matching phone row's is_valid is set to 0.

• Carrier and handset errors (21614, 30003 to 30007) and anything else are written to the Error Log.

Send a group text

1. Create a Messaging Group and add contacts. You can use Add to Group on a contact form, or select rows in the Contact list and use the Add to Group action.

2. Open Group Text Message and write a title (internal only) and the message.

3. Choose Target Groups, and optionally Exclude Groups. Anyone in an excluded group is skipped.

4. To send later, tick Schedule and set Delivery Date/Time in the future.

5. Click Schedule/Send. The document is submittable: an unscheduled one sends at once, a scheduled one gets status Scheduled.

A scheduler job (registered under the all scheduler event) sends scheduled messages once their delivery time has passed.

A contact receives the text only if every one of these is true:

• They belong to a target group and no excluded group.

• consent_sms is 1 and unsubscribed is 0.

• A phone row is flagged Is Primary Mobile.

• That row has Is Valid E164 set.

If nobody qualifies, the document is marked Error with an Info comment. Otherwise it gets a comment listing who was sent to and who failed (with the Twilio error code), and its status becomes Sent if at least one message went out. The System Manager, Newsletter Manager and Marketing Manager roles can create group texts.

Keep an Email Group in sync

A Messaging Group can link to an Email Group. When you save the group, the app adds missing member emails to the Email Group and removes Email Group members who are no longer in the messaging group. Setting Messaging Group for All Contacts in Messaging Settings adds every contact to one group whenever it is saved.

Newsletters with merge tags

The app overrides the Newsletter class from the newsletter app so each recipient gets their own copy. For every address it looks up the matching contact by email and renders your content as a Jinja template. These tags are available:

{{ first_name }}
{{ last_name }}
{{ full_name }}
{{ company_name }}
{{ designation }}
{{ gender }}
{{ salutation }}

The newsletter form lists the tags as help text beside the content. They work in Rich Text, Markdown and HTML newsletters, and in test emails.

If an address does not match a contact, full_name falls back to the part of the address before the @, and the other tags render empty. If tags come out blank for someone you know, check that their email is saved on a Contact, spelled the same way.

Newsletters are queued one recipient at a time, and the usual unsubscribe link and read tracking from the newsletter app still apply.

Use the unified inbox

Open the Communication list. It defaults to the Chat view, which lives at /desk/communication/view/chat on Frappe v16 (an /app/communication/view/chat link redirects there). The Messaging workspace adds shortcuts to the inbox, new group texts, new newsletters and notifications.

How conversations are built

Each conversation (a "room") is one outside address in one medium. Room IDs look like this:

SMS:+15555550123
Email:jane@example.com

SMS conversations are keyed on the phone number. Email conversations are keyed on the sender, or the first recipient for mail you sent. Contacts are matched by number or address to show names and avatars. The list puts conversations whose last received message has not been replied to or closed first, then sorts by date.

You can filter by medium (All, Email, SMS), search across subject, body, sender, recipients, phone number and contact names, and mark conversations read. The app also supports archiving a conversation (all its messages set to Closed) and deleting one, which deletes every message in it, so use it sparingly.

New messages arrive over Frappe's realtime channel, so open inboxes update without a refresh. Browsers can also opt in to Web Push notifications from a button in the page header; the VAPID keys for that are generated from a button in Messaging Settings.

Replying

Pick a conversation and type. What happens depends on the medium:

• SMS: a Communication is created and sent through the Twilio path above. Its delivery status becomes Sent or Error.

• Email: the message is queued with frappe.sendmail, with your Email Signature from your User record (or your name if none is set), a quoted copy of the message you replied to, and the default incoming Email Account as reply-to.

Replying marks the latest received message in the thread as Replied.

API and development

Everything the view needs is whitelisted under messaging.messaging.api.chat.api: get_rooms, get_messages, send_message, mark_messages_seen, search_rooms, get_unread_count, archive_room, delete_room and get_current_user. The front end is TypeScript and Vue built on vue-advanced-chat. To work on it, run yarn install in the app directory, then bench build --app messaging. The repository also has yarn typecheck and yarn watch scripts.

Troubleshooting

• No inbound texts appear. Confirm the webhook URL, the auth token in Messaging Settings, and that the site is reachable over HTTPS with the scheme Twilio called. Signature failures are rejected by the handler.

• Group text says no eligible recipients. Check consent, unsubscribed, the primary mobile flag and Is Valid E164 on the contact.

• A contact stopped receiving texts. Look for an Info comment from an opt-out, or a Twilio 21610 entry in the Error Log.

• Merge tags are blank. The recipient is not matched to a Contact by email.