I have also played with specialized machine learning (ML) acceleration hardware for computer vision with projects like Google's AIY Vision Kit and Intel's Movidius on the client side. Edge ML is powerful, but it has performance limits compared to scalable server-side approaches where you can add more and bigger processors. In a previous project using WebRTC and the Tensorflow Object Detection API, I looked at server-side computer vision (CV) using generic CPU resources. It would have been much more effient to run this with GPU acceleration, or even TPU acceleration, but that requires actually having that specialized hardware. Of course you can access that in the cloud, but with addition cost and platform dependencies. Wouldn't it be nice to get hardware acceleration without specialized hardware?

Intel offers native hardware acceleration on its CPU chips through various projects. I have been wanting to play around with the Intel Open WebRTC Toolkit (OWT) server since they demoed some of its CV capabilities at Kranky Geek, but these kinds of projects typically require a serious time commitment. Fortunately Intel decided to sponsor some of my time to install and evaluate the project for a webrtcHacks post.

Check that out here:
https://webrtchacks.com/accelerated-computer-vision-inside-a-webrtc-media-server-with-intel-owt/

Like nearly every annual conference, the CommCon event went virtual this year. I joined to give an update on how COVID-19 has impacted the RTC industry and what it might mean after pandemic dies down. In particular, I wanted to explore the Zoom phenomenon and why a company that seems to do everything contrary to what the larger WebRTC community has so much success.

In this talk, I cover:

• Major trends and what might be next in 4 domains:

       1. Remote work

       2. Telemedicine

       3. Education

       4. Developers

• Some new quantitative research on WebRTC in open source
• Did Zoom beat WebRTC?
• Some Hacker News sentiment analysis on Zoom vs. WebRTC
• Some thoughts on what RTC should be working on next

On that last bullet, I am encouraged to see some good video communication innovations happening already. For example,  Microsoft Teams released their Together mode days after I recorded my session.

Check out the video here:

These are chaotic times for most businesses and contact centers are no exception. Depending on the business, the contact center may have increased call volumes and/or new staffing challenges with more dependency on home agents, shifting work schedules, and even agent furloughs in some cases. Automation is increasingly becoming a necessity just to maintain pace.

Voximplant asked me to join a webinar to discuss how conversational IVRs - i.e. voicebots hooked into a phone - can help with process automation without killing customer satisfaction. I discuss how modern voice assistant technology is democratizing conversational IVR technology, making them more effective and lifelike.

Check out the webinar recording below.

I teamed up with Dave Horton to do the #TADHack this past weekend. Although I have helped as a sponsor and even hosted a TADHack in Boston, it has been many years since I last entered as a participant (see that one here
). This year a subtheme was Intelligent Agents. That is an active area I researching and blogging on over at cogint.ai so I was eager to try some things in that domain.

We ended up making a tool for small businesses that helps them rebook customer appointments using:

• Dialogflow voicebot,
• drachtio.org as a SIP server and gateway and
• Simwood Group for phone numbers and SIP trunk connectivity.
• Google Calendar and Sheets as the UI

The business just needs to connect a Google Calendar and enter thier customer numbers into a Google sheet and the bot will auto-dial them and attempt to reschedule. Call status and disposition are updated back to the sheet. The customer can ask for a callback at a better time or request to talk to a human and get transferred if needed.

See out our winning submission here:

Experimenting with running computer vision locally in the browser using TensorFlow.js has been on my bucket list to play with as part of my regular AI in RTC research. The recent warnings to avoid touching your face got me thinking there should be some kind of webcam monitor that warns you when you touch your face.

I did a quick websearch and found one site that does this, but it requires user training on ever refresh and wasn't all that accurate. So, I decided to give the TensorFlow.js with the BodyPix model library a try as a weekend project.

Check out the results of my experiment at facetouchmonitor.com.

It's not perfect, but I was amazed how well it worked without a whole lot of effort - all in the browser. All the code is available at github.com/webrtchacks/facetouchmonitor. I shared some highlights on how it works and an alternative approach at webrtcHacks: https://webrtchacks.com/stop-touching-your-face-with-browser-tensorflow-js/

In addition to their annual conference, the team behind FreeSWITCH runs a regular session on YouTube called ClueCon Weekly. I joined a couple of weeks ago for a discussion with David Duffett. We covered a lot in the ~30 min session, including:

• How I got into comms/telephony
• How WebRTC is doing
• Why most IVR's suck
• What I am covering (AI in RTC) over at cogint.ai

You can watch the video here:

Earlier this month I joined Arin Sime of WebRTC.ventures for the regular WebRTC Live! webmiar series. I talked about the state of WebRTC and recent WebRTC Hacks blog posts.

Check it out here:

WebRTC Live #38 - "Updates from webrtcHacks," Chad Hart, webrtcHacks.com from arinsime on Vimeo.

I generally produce longer, indepth posts over at webrtcHacks. However, that format doesn't work for all posts and there are many topics we want to cover. So, we're trying out shorter format for some posts. Let me know what you think!

I cover Perfect Negoition in the first post in this new series. Perfect Negotiation is one of the many multi-faceted efforts taking place behind the scenes to help improve WebRTC. In this case it is a set of spec and implementation improvements designed to make connectivity more reliable. Check out the post for details: https://webrtchacks.com/min-duration-series-part-1-perfect-negotiation/

WebRTC has a new browser – kind of. Microsoft GA’ed its “new” Edge browser based on Chromium – commonly referred to Edgium. On the surface one less WebRTC engine to work with should make life easier for developers, but does the reality match the ambition?

I did a review of Edgium's WebRTC features to see how it compares to Chrome and what differences there are. The short answer is almost none. Check out my full review for details over at webrtcHacks:
https://webrtchacks.com/edgium-webrtc-review/

I have been researching the use of modern voice-based bot platforms - i.e. voicebots - for use in conversational IVR applications over at my cogint.ai blog.  I also covered this application and the gateway needed too connected VoIP systems to bot platforms in the KGR AI in RTC report. Communications Platform as a Service (CPaaS) vendor Voximplant was the first vendor with a clould gateway for Dialogflow. They hired me to write a whitepaper on this topic.

The paper covers:

• The benefits of voice bots vs. traditional IVR technologies
• Why use Dialogflow specifically with a voicebot IVR?
• How to connect Dialogflow to phone calls?
• Review of Voximplant's Dialogflow Connector

You can download the whitepaper from Voximplant's site: https://voximplant.com/promo/dialogflow_201912

My last several cogint.ai posts have looked at the use of voice-enabled bots – aka voicebots – for use in Interactive Voice Response (IVR) and other telephony applications. I continued on this theme with a review of another voicebot telephony connectivity option - AudioCodes' Voice.AI Gateway. The AudioCodes Voice.AI gateway has some unique qualities compared to other RTC-bot gateway solutions on the market - notably its multiplatform approach and the fact it is offered as a managed service.

I ran some test againts the Voice.AI gateway with Dialogflow and looked at their pricing. Read the full post here: https://cogint.ai/audiocodes-voice-ai-gateway-review/

Kranky Geek is a technology event series focused on various aspects of Real Time Communications (RTC). WebRTC is always a major topic and we have been adding more talks on how AI and ML intersect with RTC too. We have held events all over the world and this year held our event at Google in San Francisco, our 6th time at that particular location.

Outside my shared responsibilities in finding highly relevant speakers, providing coaching for the talks, and helping get people to the event, I also spoke during two sessions this year.

Introduction & Welcome

Tsahi and I usually split the introduction. This year I spoke briefly about how my analysis from my WebRTC on Github analysis webrtcHacks post shows that WebRTC is far from stale with major new projects emerging.

Check that video out here:

WebRTC Browser Roundtable

I was also very excited to host our first-ever WebRTC Round Table with all the major browsers - Chrome, Firefox, Safari, and Edge. The panelists included represented by lead developers and Product Managers on the actual browser teams. The session started with each panelist spending a few minutes on what's new and what's coming. For the 2nd hald of the session I moderated an interactive discussion covering mobile, PWA, security, and ML among other topics that came-up.

See that session here:

So much more

We had 14 unique sessions featuring 17 speakers (other than me & Tsahi). Head over to krankygeek.com to see the full agenda or watch the playlist here:
https://www.youtube.com/playlist?list=PL4_h-ulX5eNdJljPgBWAgD8l49YOjeMoc

The Illinois Institute of Technology Real Time Communications (IIT-RTC) conference is an academically inspired conferences I frequently attend. This year I moderated a panel with Chris Wendt of Comcast, Vijay Gurbani or Vail Systems, and Varun Signh of callstats.io.

I introduced the topic and provided a framework from my past resarch in this area. Each of the panelists then spent a few minutes talking about their individual Machine Learning (ML) use cases. We wrapped up in a discussion covering the following questions:

• What ML methods have you found most effective?
• What are your preferred ML libraries and frameworks?
• What is the best way to train your development teams developing with ML?
• How do you think advances in ML hardware - both at the edge and in the cloud - will impact RTC applications?
• Is Google (or Amazon or Microsoft, etc.) going to come along and make all of this super-easy? What does that mean for your efforts?

You can see the entire recorded session here:

I am asked all the time what the most popular WebRTC projects are. I tried to address this question a few years ago in Data Nerding with WebRTC GitHub Data where I used the GitHub dataset on BigQuery to filter out WebRTC repos. I did an update to this analysis and found:

• WebRTC is doing just fine - its growth has more or less remained consistent with github overall
• There have been meaningful shifts in the top repos:
  • pion gets the crown for the most popular project by most measures
  • Janus wins among WebRTC media servers
  • Coturn appears strongly now that I included TURN server terms in the analysis
  • Several Microsoft projects made the top lists
  • react-nativev-webrtc leads on mobile/hybrid frameworks
  • startrtc is a new but big project out of China

See the full post here: https://webrtchacks.com/webrtc-open-source-most-popular/

Contact me if you are interested in additional analysis on this dataset.

I have been very interested in how modern voice assistants can be used with Real Time Communications even before I started my AI in RTC research. I partnered with
Emiliano Pelliccioni at webrtc.ventures on a research project to understand how to build build conversational, voice-driven bot interfaces that could replace traditional Interactive Voice Response systems. We had previously evaluated several different voice assistant interfaces and Natural Language Understanding (NLU) frameworks and we decided on Dialogflow for this project.

Our single summary post of our findings quickly grew too long, so we decided to break our findings up into a series of posts over at cogint.ai. This series includes:

1. Methods for Connecting a Phone Call to Dialogflow
2. SignalWire’s Dialogflow Connector: 2019 review
3. VoxImplant's Dialogflow Connector: 2019 review
4. Adding SMS to your Voicebot IVR
5. Building a Multi-business Voicebot IVR

I also did a presentation at ClueCon summarizing this series:

Here are those slides:

If you care about this topic, you might also like my original How to make a dial-in IVR with Dialogflow tutorial.

Of course I am excited to consult on this topic too.