9ac9c2d9172a45239c85dd6b31b7e3fe by 2023

Author:2023
Language: eng
Format: epub
Published: 2023-09-30T17:13:32+00:00

implemented at 16-bits. Some platforms redefine it as 32-bits and in this form it is equivalent to UCS4 and hence full Unicode. However, most platforms, Windows in particular, have stuck with wchar_t as 16 bits.

The C API provides a single function which will create a Python Unicode string from a wchar_t array:



PyObject *PyUnicode_FromWideChar(const wchar_t *w,

Py_ssize_t size)

If you pass size=-1 the function works out the size of the NULL-terminated wchar_t string. If wchar_t is 16-bit this function will detect and convert surrogate pairs and hence it treats the wchar_t string as being UTF-16

encoded.

There are two functions which encode a Python Unicode string as a wchar_t array:



Py_ssize_t PyUnicode_AsWideChar(PyObject *unicode,

wchar_t *w, Py_ssize_t

size)



wchar_t *PyUnicode_AsWideCharString(PyObject *unicode,

Py_ssize_t *size)

The difference between them is that the first needs a wchar_t buffer passed in and the second creates the buffer for you. Of course, you have to use PyMem_Free() to free the generated buffer. If you set size to NULL a full NULL

terminated string is returned with no embedded NULLs. If wchar_t is 16-bit and the Unicode character cannot be represented in 16-bits then a surrogate pair is generated, i.e. the resulting wchar_t string is encoded as UTF-16. If wchar_t is 32-bit then these functions reduce to a copy operation.

Code Pages

Before Unicode systems displayed characters beyond the basic ASCII using

“code pages”. A code page provides a range of 256 characters that can be displayed. Characters were selected using extended ASCII, i.e. the first 128

characters are standard ASCII and the custom characters correspond to codes above 127. So to display a character that is not part of the ASCII character set you first select a code page that has it and then use its code in the code page. Of course, if the machine has a different code page selected then the code you use will not correspond to the character you want. Each code above 127 corresponds to a range of different characters depending on the code page that is active.

146

You can make use of code pages on Windows machines using the Code Page’s codec. For example:

PyObject *myBytes = PyUnicode_AsEncodedString(myString,

"cp1252", NULL);

encodes the string according to cp1252, Code Page 1252, i.e. the Latin Code page for Windows and stores a pointer to the result as a bytes object in myBytes.

A function to encode a string into the Latin code page is:

static PyObject *string5(PyObject *self, PyObject *args)

{

PyObject *myString = PyUnicode_New(1, 1114111);

PyUnicode_WriteChar(myString, 0, 0x2020);

PyObject *myBytes = PyUnicode_AsEncodedString(

myString, "cp1252", NULL);

char *buffer;

Py_ssize_t len;

PyBytes_AsStringAndSize(myBytes, &buffer, &len);

printf("%X
", buffer[0]);

printf("%s
", buffer);

Py_DECREF(myBytes);

return myString;

}

Unicode character 0x2020 is † i.e. a dagger symbol and this doesn’t occur in the ASCII code but it is character 0x86 in the Latin code page. If you run the program above under Windows you will probably see:

86

Ã¥

The 86 corresponds to the code for the dagger in the Latin code page which is what was requested. The character you see printed depends on what code page the editor’s terminal is using. In the example above it is the Windows terminal which uses code page 850 Latin-1 by default and code 0x86 is

“Lower case a with ring above”.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.