Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failing locale tests on Apple platforms with the RU locale #45084

Closed
ldionne opened this issue Apr 29, 2020 · 7 comments
Closed

Fix failing locale tests on Apple platforms with the RU locale #45084

ldionne opened this issue Apr 29, 2020 · 7 comments
Assignees
Labels
bugzilla Issues migrated from bugzilla confirmed Verified by a second party libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.

Comments

@ldionne
Copy link
Member

ldionne commented Apr 29, 2020

Bugzilla Link 45739
Version unspecified
OS All
CC @arichardson,@mclow

Extended Description

The following tests are failing on Apple platforms:

test/std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_ru_RU.pass.cpp b/libcxx/test/std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_ru_RU.pass.cpp
test/std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_ru_RU.pass.cpp b/libcxx/test/std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_ru_RU.pass.cpp

We should figure out why and fix them.

@ldionne
Copy link
Member Author

ldionne commented Apr 29, 2020

assigned to @ldionne

@ldionne
Copy link
Member Author

ldionne commented Apr 29, 2020

Here's a reproducer for one of these failing tests. I suspect they are all the same failure under the hood:

$ clang++ repro.cpp -g -std=c++2a -nostdinc++ -I libcxx/include -isysroot $(xcrun --show-sdk-path) -I libcxx/test/support -L build/lib -nostdlib++ -lc++ -Wl,-rpath,$PWD/build/lib -o a.out && ./a.out

$ cat repro.cpp
#include
#include
#include
#include
#include "test_macros.h"
#include "test_iterators.h"

#include "platform_support.h" // locale name macros

typedef std::money_get<char, input_iterator<const char*> > Fn;

struct my_facet : Fn {
    explicit my_facet(std::size_t refs) : Fn(refs) {}
};

int main(int, char**) {
    std::ios ios(0);
    std::string loc_name(LOCALE_ru_RU_UTF_8);
    ios.imbue(std::locale(ios.getloc(),
                          new std::moneypunct_byname<char, false>(loc_name)));
    ios.imbue(std::locale(ios.getloc(),
                          new std::moneypunct_byname<char, true>(loc_name)));
    ios.imbue(std::locale(ios.getloc(),
                          new std::moneypunct_byname<wchar_t, false>(loc_name)));
    ios.imbue(std::locale(ios.getloc(),
                          new std::moneypunct_byname<wchar_t, true>(loc_name)));

    const my_facet f(1);
    {   // zero, showbase
        std::string v = "0,00 RUB ";
        showbase(ios);
        typedef input_iterator<const char*> I;
        long double ex;
        std::ios_base::iostate err = std::ios_base::goodbit;
        I beg(v.data());
        I end(v.data() + v.size());
        I iter = f.get(beg, end, true, ios, err, ex);
        assert(iter.base() == v.data() + v.size());
        assert(err == std::ios_base::eofbit);
        assert(ex == 0);
        noshowbase(ios);
    }

    return 0;
}

@arichardson
Copy link
Member

I also ran into this while testing on FreeBSD. It appears that the problem is that the space after "RUB " is not consumed on Apple and FreeBSD.

I can upload a patch for this shortly. Would you prefer

a)

#if defined(FreeBSD) || defined(APPLE)
// FreeBSD's and Apple's libc do not consume the space.
assert(iter.base() == v.data() + v.size() - 1);
assert(err == std::ios_base::goodbit);
#else
assert(iter.base() == v.data() + v.size());
assert(err == std::ios_base::eofbit);
#endif

Or b)

Changing e.g. "-1 234 567,89 RUB " to "-1 234 567,89 RUB". I'm not sure if the space is important, but removing it fixes the test for me.

@arichardson
Copy link
Member

The fr_FR tests seem to use "1 234 567,89 EUR" without the trailing space, so I guess that means omitting it is fine?

@ldionne
Copy link
Member Author

ldionne commented Nov 10, 2020

I think I would do b), so that the tests are as streamlined as possible.

TBH, I don't understand why there's subtle differences like:

#if !defined(APPLE_FIXME)
{   // zero, showbase
    std::string v = "0,00 RUB ";
    showbase(ios);
    typedef input_iterator<const char*> I;
    long double ex;
    std::ios_base::iostate err = std::ios_base::goodbit;
    I iter = f.get(I(v.data()), I(v.data() + v.size()),
                                        true, ios, err, ex);
    assert(iter.base() == v.data() + v.size());
    assert(err == std::ios_base::eofbit);
    assert(ex == 0);
    noshowbase(ios);
}
#endif
{   // negative one, showbase
    std::string v = "-0,01 RUB ";
    typedef input_iterator<const char*> I;
    long double ex;
    std::ios_base::iostate err = std::ios_base::goodbit;
    I iter = f.get(I(v.data()), I(v.data() + v.size()),
                                        true, ios, err, ex);
    assert(iter.base() == v.data() + 6);
    assert(err == std::ios_base::goodbit);
    assert(ex == -1);
}

Why do we check for eofbit when parsing 0,00 RUB, but goodbit when parsing -0,01 RUB?

I would try to remove the trailing space and see whether that still works. If you upload a patch on Phabricator, that'll also run the tests on Linux and macOS automatically, so you can experiment that way.

Thanks for looking into it.. these locale APIs are terrible. :-)

@llvmbot llvmbot transferred this issue from llvm/llvm-bugzilla-archive Dec 10, 2021
@llvmbot llvmbot added the confirmed Verified by a second party label Jan 26, 2022
@mstorsjo
Copy link
Member

I ran into this issue, while poking those test files, and I went ahead and tried to fix it - see https://reviews.llvm.org/D120316.

I also agree that we should remove the trailing space there; that matches other similar tests. This test seemed to be unusually untested in the sense that it was XFAILed in essentially all configurations in our current CI except for Apple platforms, and on Apple platforms, these bits were ifdeffed out.

On top of that, I made https://reviews.llvm.org/D120317 which makes the test a bit more flexible, so that it passes on Glibc, FreeBSD and Windows too.

mstorsjo added a commit that referenced this issue Feb 24, 2022
…forms

This fixes issue #45084 (https://llvm.org/PR45739).

Remove unnecessary trailing spaces after the "RUB" international
currency symbol (and after the plain number in some parts of the
put_long_double test).

Both of these test files are
`XFAIL: netbsd || linux || LIBCXX-WINDOWS-FIXME`, and then have some of
their test cases commented out when `__APPLE__`. This patch comments-in
those test cases and adjusts them all to work on Apple, while leaving the
test `XFAIL`ed on NetBSD, Linux, and Windows.

Differential Revision: https://reviews.llvm.org/D120316
@mstorsjo
Copy link
Member

This issue should be fixed now by 5333732.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugzilla Issues migrated from bugzilla confirmed Verified by a second party libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Development

No branches or pull requests

4 participants