Universal Acceptance: Conformance Testing of Libraries and Languages
This set of pages describes the work done by Cofomo on testing Programming languages and libraries against the Universal Acceptance compliance requirements. Three series of tests were executed: one in 2018, one in 2020 and another on 2021. This page describes the latest one.
2021 Test Run
The report [pdf] contains the information about the methodology used, the tested languages and libraries, conformance level of the libraries and recommendations for developers.
There is also a PowerPoint presentation [pptx].
The UA conformance detailed results are breakdown by platforms:
Bug Reports
Here are the bug reports for each of the non-compliant tested libraries:
| Language | Platform | Library | Report | Resolution | Notes |
|---|---|---|---|---|---|
| Swift | iOS | MessageUI | StackOverflow report | No answer yet | Bug has been reported on Apple’s bug reporting tool but link is private. See iOS note below |
| Swift | iOS | URLSession & Alamofire | StackOverflow report | Being discussed | Bug has been reported on Apple’s bug reporting tool but link is private |
| Swift | iOS | URLSession & Alamofire | Bug report on Swift bugtracker | Assigned to a maintainer | |
| PHP | Windows | Bug report on RFC6531 compliance | Changed to feature request | There is very few chances that it would be implemented | |
| PHP | Windows | cURL | Bug report on IDNA 2008 compliance | This is a Windows issue | The library used for IDN conversions is a windows version that is only IDNA2003 compliant |
| PHP | Windows | cURL | Bug report on IDN 2008 compliance | No answer yet | Windows feedback hub report, as PHP maintainers stated that the issue came from Windows |
| PHP | Windows & Linux | intl | Bug report on IDNA 2008 compliance | Suspended as this is an ICU issue | Unicode ICU library implements IDNA2008 according to UTS #46, see their documentation |
| PHP | Windows & Linux | PHP Mailer | Bug report for IDNA 2008 compliance | Pull request submitted and merged upstream | |
| PHP | Windows & Linux | Symfony HttpClient | Bug report on IDNA 2008 compliance | Pull request submitted and merged upstream | |
| PHP | Windows & Linux | Symfony Mailer | Bug report on IDNA 2008 compliance | Pull request submitted and merged upstream | |
| PHP | Windows & Linux | Symfony Mailer | Bug report on RFC5322 compliance | Being discussed | |
| PHP | Windows & Linux | Symfony Mailer | Bug report on RFC6531 compliance | No answer yet | |
| Kotlin | Android | Jakarta Mail | Bug report on RFC6531 compliance | No answer yet | |
| Kotlin | Android | HttpUrlConnection | Bug report on IDNA 2008 compliance | Transmitted to the engineering teams | |
| Kotlin | Android | OkHttp | Bug report on IDNA 2008 compliance | Closed after Q&A | Maintainer closed it “as the strictness of IDNA 2008 is likely to cause more visible issues than this solves, particularly as this isn’t uniformly supported or implemented by clients and servers”. While major actor in the industry will stick to IDNA 2003 (e.g. Chrome browser) this is not likely to change. |
| Kotlin | Android | Fuel | Bug report on IDNA 2008 compliance | Pull request submitted and pending to be merged upstream | |
| Kotlin | Android | Apache HttpClient | Apache HttpClient | No answer yet |
iOS note
The bug report on MessageUI & URLSession points to a stackoverflow, since Apple’s proprietary bug reporting tool (called Feedback Assistant) maintains reports private between developers and the company.
The bug is registered at both place: stackoverflow & Feedback Assistant for the benefits of the community:
For URLSession & Alamofire, the bug seems to come from the open-source version of the Foundation framework. We thus log the bug also on the Swift open-source bug tracker.
Windows note
We log only two bug reports for two PHP libraries on Windows. The other libraries behave the same way on Linux & Windows.
On Linux, the PHP mail extension uses sendmail which provides for the SMTPUTF8 extension since version 8.17. On Windows, the PHP mail extension is the Mail Transfer Agent (MTA) and would be responsible to send the SMTPUTF8 flag.
The lib cURL contained in the PHP Windows binaries uses the “windows.h” IdnToUnicode/IdnToAscii” functions which are known to be only IDNA2003 compliant. We opened a bug in the Windows’s Feedback Hub for that:
Android note
Volley is developed by Google and is part of Android, moreover, it uses HttpUrlConnection therefore the HttpUrlConnection bug report is sufficient to cover Volley compliance.
The same goes for Retrofit that is using OkHttp stack and utils and is maintained by the same company.
PHP Mailer note
The PHP Mailer maintainer is well aware of EAI and already got many bug reports about that. See the latest one where he mentions others therefore no bug additional bug for EAI support has been reported.