Universal Acceptance: Conformance Testing of Libraries and Languages

This set of pages describes the work done by Cofomo on testing Programming languages and libraries against the Universal Acceptance compliance requirements. Three series of tests were executed: one in 2018, one in 2020 and another on 2021. This page describes the latest one.

2021 Test Run

The report [pdf] contains the information about the methodology used, the tested languages and libraries, conformance level of the libraries and recommendations for developers.

There is also a PowerPoint presentation [pptx].

The UA conformance detailed results are breakdown by platforms:

Bug Reports

Here are the bug reports for each of the non-compliant tested libraries:

Language Platform Library Report Resolution Notes
Swift iOS MessageUI StackOverflow report No answer yet Bug has been reported on Apple’s bug reporting tool but link is private. See iOS note below
Swift iOS URLSession & Alamofire StackOverflow report Being discussed Bug has been reported on Apple’s bug reporting tool but link is private
Swift iOS URLSession & Alamofire Bug report on Swift bugtracker Assigned to a maintainer  
PHP Windows mail Bug report on RFC6531 compliance Changed to feature request There is very few chances that it would be implemented
PHP Windows cURL Bug report on IDNA 2008 compliance This is a Windows issue The library used for IDN conversions is a windows version that is only IDNA2003 compliant
PHP Windows cURL Bug report on IDN 2008 compliance No answer yet Windows feedback hub report, as PHP maintainers stated that the issue came from Windows
PHP Windows & Linux intl Bug report on IDNA 2008 compliance Suspended as this is an ICU issue Unicode ICU library implements IDNA2008 according to UTS #46, see their documentation
PHP Windows & Linux PHP Mailer Bug report for IDNA 2008 compliance Pull request submitted and merged upstream  
PHP Windows & Linux Symfony HttpClient Bug report on IDNA 2008 compliance Pull request submitted and merged upstream  
PHP Windows & Linux Symfony Mailer Bug report on IDNA 2008 compliance Pull request submitted and merged upstream  
PHP Windows & Linux Symfony Mailer Bug report on RFC5322 compliance Being discussed  
PHP Windows & Linux Symfony Mailer Bug report on RFC6531 compliance No answer yet  
Kotlin Android Jakarta Mail Bug report on RFC6531 compliance No answer yet  
Kotlin Android HttpUrlConnection Bug report on IDNA 2008 compliance Transmitted to the engineering teams  
Kotlin Android OkHttp Bug report on IDNA 2008 compliance Closed after Q&A Maintainer closed it “as the strictness of IDNA 2008 is likely to cause more visible issues than this solves, particularly as this isn’t uniformly supported or implemented by clients and servers”. While major actor in the industry will stick to IDNA 2003 (e.g. Chrome browser) this is not likely to change.
Kotlin Android Fuel Bug report on IDNA 2008 compliance Pull request submitted and pending to be merged upstream  
Kotlin Android Apache HttpClient Apache HttpClient No answer yet  

iOS note

The bug report on MessageUI & URLSession points to a stackoverflow, since Apple’s proprietary bug reporting tool (called Feedback Assistant) maintains reports private between developers and the company.

The bug is registered at both place: stackoverflow & Feedback Assistant for the benefits of the community:

Feedback Assistant's screenshot

For URLSession & Alamofire, the bug seems to come from the open-source version of the Foundation framework. We thus log the bug also on the Swift open-source bug tracker.

Windows note

We log only two bug reports for two PHP libraries on Windows. The other libraries behave the same way on Linux & Windows.

On Linux, the PHP mail extension uses sendmail which provides for the SMTPUTF8 extension since version 8.17. On Windows, the PHP mail extension is the Mail Transfer Agent (MTA) and would be responsible to send the SMTPUTF8 flag.

The lib cURL contained in the PHP Windows binaries uses the “windows.h” IdnToUnicode/IdnToAscii” functions which are known to be only IDNA2003 compliant. We opened a bug in the Windows’s Feedback Hub for that:

Feedback Hub's screenshot

Android note

Volley is developed by Google and is part of Android, moreover, it uses HttpUrlConnection therefore the HttpUrlConnection bug report is sufficient to cover Volley compliance.

The same goes for Retrofit that is using OkHttp stack and utils and is maintained by the same company.

PHP Mailer note

The PHP Mailer maintainer is well aware of EAI and already got many bug reports about that. See the latest one where he mentions others therefore no bug additional bug for EAI support has been reported.