PHP Programming - Html-encode all characters not in the current character set

Author

Html-encode all characters not in the current character set

Timothy Madden

2007-04-26, 7:58 am

Hello

Is there a function that will allow me to
output text written in utf-8 (from db for example)
if my document has

Content-Type: text/html; charset=ISO-8859-1

I mean htmlspecialchars() and htmlentities() will only convert
characters that have an associated entity defined in HTML.
I would also like to translate all non-latin1 characters using
numeric references.

ţ is for a Romanian letter, ţ, for example, and letter ţ
written in UTF-8 is not translated by htmlentities(), even if
I give the function the optional character-set argument, 'UTF-8'
(you can actually see the letter I typed if your system and your
news reader understand and can display ISO latin 2 characters,
encoded in utf-8).

I mean HTML documents can use characters in the entire UNICODE
set, even if the document source is written in ASCII for example,
by encoding any non-ASCII character with HTML entities.

Is there in PHP a function that will encode in HTML all non-ASCII
characters, or all non-latin1 characters, or all characters not in the
source character set ?

Thank you,
Timothy Madden

Vince Morgan

2007-04-26, 6:58 pm

"Timothy Madden" <terminatorul@gmail.com> wrote in message
news:4630999f$0$90266$14726298@news.sunsite.dk...
> Hello
>
> Is there a function that will allow me to
> output text written in utf-8 (from db for example)
> if my document has
>
> Content-Type: text/html; charset=ISO-8859-1
>
> I mean htmlspecialchars() and htmlentities() will only convert
> characters that have an associated entity defined in HTML.
> I would also like to translate all non-latin1 characters using
> numeric references.
>
> ţ is for a Romanian letter, t, for example, and letter t
> written in UTF-8 is not translated by htmlentities(), even if
> I give the function the optional character-set argument, 'UTF-8'
> (you can actually see the letter I typed if your system and your
> news reader understand and can display ISO latin 2 characters,
> encoded in utf-8).
>
> I mean HTML documents can use characters in the entire UNICODE
> set, even if the document source is written in ASCII for example,
> by encoding any non-ASCII character with HTML entities.
>
> Is there in PHP a function that will encode in HTML all non-ASCII
> characters, or all non-latin1 characters, or all characters not in the
> source character set ?

You may find the following link usefull;
http://au.php.net/utf8-decode
HTH
Vince

shimmyshack

2007-04-26, 6:59 pm

On Apr 26, 1:23 pm, Timothy Madden <terminato...@gmail.com> wrote:
> Hello
>
> Is there a function that will allow me to
> output text written in utf-8 (from db for example)
> if my document has
>
> Content-Type: text/html; charset=3DISO-8859-1
>
> I mean htmlspecialchars() and htmlentities() will only convert
> characters that have an associated entity defined in HTML.
> I would also like to translate all non-latin1 characters using
> numeric references.
>
> ţ is for a Romanian letter, =C5=A3, for example, and letter =C5=A3
> written in UTF-8 is not translated by htmlentities(), even if
> I give the function the optional character-set argument, 'UTF-8'
> (you can actually see the letter I typed if your system and your
> news reader understand and can display ISO latin 2 characters,
> encoded in utf-8).
>
> I mean HTML documents can use characters in the entire UNICODE
> set, even if the document source is written in ASCII for example,
> by encoding any non-ASCII character with HTML entities.
>
> Is there in PHP a function that will encode in HTML all non-ASCII
> characters, or all non-latin1 characters, or all characters not in the
> source character set ?
>
> Thank you,
> Timothy Madden

also mb_convert_encoding()

Willem Bogaerts

2007-04-27, 3:59 am

> Is there a function that will allow me to
> output text written in utf-8 (from db for example)
> if my document has
>
> Content-Type: text/html; charset=ISO-8859-1
>
> I mean htmlspecialchars() and htmlentities() will only convert
> characters that have an associated entity defined in HTML.
> I would also like to translate all non-latin1 characters using
> numeric references.

There are two terms of interest here: "character set" and "encoding"

ISO-8859-1 is an encoding that only covers a limited character set. So
there is no euro sign, for example. The Bad thing about ISO-8859-1 is
that some programs silently replace it with cp-1252, which is similar
but not exactly the same (it does have a euro sign).

> ţ is for a Romanian letter, ţ, for example, and letter ţ
> written in UTF-8 is not translated by htmlentities(), even if
> I give the function the optional character-set argument, 'UTF-8'
> (you can actually see the letter I typed if your system and your
> news reader understand and can display ISO latin 2 characters,
> encoded in utf-8).

So you want to encode characters that are NOT in the character set you
explicitly state. If you do want those characters, why do you state an
encoding that does not cover them? If you do want those characters, use
a character set that does have them (like unicode) and an encoding that
covers them (utf-8 is fairly common).

> I mean HTML documents can use characters in the entire UNICODE
> set, even if the document source is written in ASCII for example,
> by encoding any non-ASCII character with HTML entities.

Are you sure about that?

> Is there in PHP a function that will encode in HTML all non-ASCII
> characters, or all non-latin1 characters, or all characters not in the
> source character set ?

The htmlentities function does have an encoding parameter, but you have
already used that. As for numeric entities, I expect them to be
encoding-specific.

Best regards,
--
Willem Bogaerts

Application smith
Kratz B.V.
http://www.kratz.nl/

Timothy Madden

2007-04-27, 6:58 pm

Willem Bogaerts wrote:
>
> There are two terms of interest here: "character set" and "encoding"
>
> ISO-8859-1 is an encoding that only covers a limited character set. So
> there is no euro sign, for example. The Bad thing about ISO-8859-1 is
> that some programs silently replace it with cp-1252, which is similar
> but not exactly the same (it does have a euro sign).
>
>
>
> So you want to encode characters that are NOT in the character set you
> explicitly state. If you do want those characters, why do you state an
> encoding that does not cover them? If you do want those characters, use
> a character set that does have them (like unicode) and an encoding that
> covers them (utf-8 is fairly common).
>
>
> Are you sure about that?
>
>
> The htmlentities function does have an encoding parameter, but you have
> already used that. As for numeric entities, I expect them to be
> encoding-specific.
>
> Best regards,

As I know ISO-8859-1 is a set (of characters).

As you can see in the official HTML 4.01 specification
http://www.w3.org/TR/html401/charset.html#h-5.1
that all HTML documents use UCS defined by ISO10646, which is
identical to UNICODE.

Numeric character references can be used whatever encoding
you chose for your document source, and they always refer to
characters in UCS by their code position.

Timothy Madden,
Romania

Timothy Madden

2007-04-27, 6:58 pm

shimmyshack wrote:
> On Apr 26, 1:23 pm, Timothy Madden <terminato...@gmail.com> wrote:
[...][color=darkred]
>
> also mb_convert_encoding()
>

Actually I think mb_encode_numericentity() is the function I need.

mb_convert_encoding() will just re-encode a string from one encoding
to another, but a Latin-1 source simply can not include Latin-2
characters no matter what encoding I chose. I need numeric character
references defined by HTML for that.

Anyway I think mb_encode_numericentitiy() will work, I just need to know
how to create a map of code-point areas for it to work, and I don't
quite understand how such an area is defined.

Thank you,
Timothy Madden,
Romania

Sponsored Links