EncoderFallbackException when writing characters not available in the specified encoding with XMLWriter to a Stream - feature or bug?

I've stumbled upon unexpected behavior of the .NET 2.0 System.Xml.XmlWriter
class when using it to write data to a binary stream (System.IO.Stream). If
the amount of data is less than a certain value (which varies depending on
the data being written), characters not available in the encoding specified
in the Encoding property of the XmlWritterSettings instance used to create
the XmlWriter are being written to the resulting XML document as character
entities, which, despite not being able to provide an exact link to a
specific part of the documentation, I would call the expected behavior. For
a higher amount of data, either implicit or explicit flush of the XmlWriter
buffer causes a System.Text.EncoderFallbackException with the message
"Unable to translate Unicode character \uXXXX at index XX to specified code
page.", with the character indicated being one not available in the
destination encoding. Documentation of the Encoding property of the
XmlWriterSettings says "An exception is thrown when the Flush method is
called if any encoding errors are encountered.", but the Encoding instance
being used in the XmlWriterSettings instance used to create XmlWriter would
not cause the exception - it has an
System.Text.InternalEncoderBestFitFallback instance as the value of the
EncoderFallback property, which is replaced with
System.Xml.CharEntityEncoderFallback in the created XmlWriter instance.

An example stack trace is as follows:

System.Text.EncoderFallbackException: Unable to translate Unicode character
\u2014 at index 92 to specified code page.
   at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknown,
Int32 index)
   at System.Xml.CharEntityEncoderFallbackBuffer.Fallback(Char charUnknown,
Int32 index)
   at System.Text.EncoderFallbackBuffer.InternalFallback(Char ch, Char*&
   at System.Text.SBCSCodePageEncoding.GetBytes(Char* chars, Int32
charCount, Byte* bytes, Int32 byteCount, EncoderNLS encoder)
   at System.Text.EncoderNLS.Convert(Char* chars, Int32 charCount, Byte*
bytes, Int32 byteCount, Boolean flush, Int32& charsUsed, Int32& bytesUsed,
Boolean& completed)
   at System.Text.EncoderNLS.Convert(Char[] chars, Int32 charIndex, Int32
charCount, Byte[] bytes, Int32 byteIndex, Int32 byteCount, Boolean flush,
Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.Xml.XmlEncodedRawTextWriter.EncodeChars(Int32 startOffset,
Int32 endOffset, Boolean writeAllToStream)
   at System.Xml.XmlEncodedRawTextWriter.FlushBuffer()
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc,
Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at XmlWriterEncoderFallbackExceptionTest.Program.Main(String[] args)

An example code to reproduce the problem is as follows:

using System;
using System.Text;
using System.Xml;
using System.IO;

namespace XmlWriterEncoderFallbackExceptionTest
    class Program
        static void Main(string[] args)
            //Encoding enc = Encoding.UTF8;
            Encoding enc = Encoding.GetEncoding("iso-8859-2");
            //Encoding enc = Encoding.ASCII;

            string u2014 = "any string with characters not available in the
                + "specified encoding \u2014 like ISO-8859-2, for example";

            MemoryStream ms = new MemoryStream();

            XmlWriterSettings set = new XmlWriterSettings();
            set.Encoding = enc;

            XmlWriter xwr = XmlWriter.Create(ms, set);


                for (int i = 0; i < 1000; i++)
                    // causes the said exception if (presumably) the
                    // buffer is implicitly flushed

                // causes the said exception after a certain amount of data
                // has been written, if the WriteString call didn't do so
                // earlier


                // implicitly flushes the buffer thus may also cause the
                // exception, but not specifically in this example program
            catch (EncoderFallbackException efe)

            byte[] result = ms.ToArray();
                result.Length > 512 ?
                enc.GetString(result, result.Length - 512, 512) :

Is there a problem in my code or is it an issue in the .NET Framework 2.0
(System.Xml.CharEntityEncoderFallback class specifically, I'd guess) itself?

Janusz Nykiel

cynic3734 (1)
3/1/2007 6:53:38 PM
dotnet.xml 7266 articles. 0 followers. Follow

0 Replies

Similar Articles

[PageSpeed] 29


Similar Artilces:

FYI Office 2003 Updates Now Available
Visit http://office.microsoft.com and click on Check for Updates Word 2003 Update: KB834702 (Finnish) / Word 2003 Update: KB834702 for Multilingual User Interface Pack (English). This update fixes an issue with the user interface for the Finnish version of Word 2003. The OK button is missing from the Table Properties dialogue box. Access 2003 Danish Update: KB834013 (Danish) / Access 2003 Danish Update for Multilingual User Interface Pack: KB834013 (English). This update from Microsoft prevents a potential error. If this error occurs, the three Danish characters �, � and � do not appear cor...

Resolove a couple bugs in 2000?
OK, these may not be bugs, but they are varments if I created them, or if they are a virus.LOL I have a file that is using "Get External Data" to retrieve data from an Intranet database. I use one sheet to hold the data from the web site. On another sheet, I have my formulas referencing the first sheet. On a final sheet, I basically display what ever it is that that data represents in easy to read terms, formats, etc. Two problems can occur using the Get External Data function. Excel prompts a message box requiring acknowledgment when ever the external data is not avaliable. The ...

specifying the encoding attribute explicitly
Hello, Using code below to create an xml file, how do I specifiy utf-8 encoding in the root element, ie create "<?xml version="1.0" encoding="utf-8"?> Thanks! ------------------------------------------- Dim xmlWriter As New XmlTextWriter("C:\inetpub\wwwroot\rss2\20061019\test.xml", Nothing) xmlWriter.Formatting = Formatting.Indented With xmlWriter .WriteStartDocument() .WriteStartElement("rss") .WriteAttributeString("version", "2.0") .WriteStartElement(&...

Bug in OE 2003 after last Windows Update
After last updates (April 11), when I want to attach a file resident in "Shared Documents", Outlook hangs (and I must kill it) when I click on that folder. My OS is Italian WinXP Pro SP2 fully patched. Office is 2003 SP2 Italian fully patched. Thanks. Maurizio Bellantone Outlook hangs even when I want to save in "Shared Documents". Thanks. Maurizio Bellantone "Maurizio BELLANTONE" <bellantoneCHIOCCIOLAlibero.it> ha scritto nel messaggio news:OONrBBcXGHA.1196@TK2MSFTNGP03.phx.gbl... > After last updates (April 11), when...

Encoding for Publisher 2003
Which encoder should be used for publishing web pages to the internet. When we installed Pub2003, it defaulted to Western European. Is there a site that explains the difference between these encoders, i.e. Western European vs. US ASCII? Thank you. LoBo Designs Leave it at Western European. That setting should not need adjusting. If you should happen to use some odd symbol and font that doesn't render correctly then setting it to Unicode would be required. In your IE browser go to View, Encoding... you will find it at Western European as well ( or possibly utf8). Please use our web fo...

Outlook2003 MAJOR Bugs to reduce productivity
I have found three MAJOR bugs so far in Outlook2003, The first is the Junk Email filter that is part of the program will not work on any sub-folders under the Inbox, it will only work on the Inbox. The Second is that Outlook 2003 will not function properly on a computer that has "WinFaxPro 10.02 for Windows XP" installed. The third bug is that the macro functions do not work 100% of the time. The first time i reply to an email using any of my macros they do not always work, i have to close the email and open it back again before it will work properly. it has to do with line c...

Read/Write Excel under IIS .NET
Hi, I have configured two machines. One is W2K PRO and other is W2K server. Both has the Office server automation set up as suggested by Microsoft. I wrote a web app to create excel file and and read excel file. There are two problems on the W2K server but not the W2K PRO. W2K PRO using HTTP. W2K Server using HTTPS. 1. The excel process does not exit. 2. I got an "unspecified error" when try to read a file from the web client's local disk. Any suggestion for troubleshooting? ...

redistributable for VC++ 2005 SP1 available?
I need a setup which installs all the runtime files needed for a VC++ 2005 SP1 (x86) program. On the Microsoft homepage I only found such a setup for VC++ 2005, dated 4/10/2006, so it seems to be without SP1. Someone told me to download and install SP1 on that client, but I don't know if this works if no Visual Studio products are installed and furthermore I don't want to download and install a 430MB setup package. (BTW: This has to be installed later on 2000 clients, when the tests complete.) Will this be released later or did I just not find the link? Eric On Fri, 2 Feb 200...

ANN: Database Workbench 4.0.3, the multi-DBMS IDE now available!
Ladies, gentlemen, Upscene Productions is proud to announce the next version of the popular multi-DBMS development tool: " Database Workbench 4.0.3 Pro " This release includes the FREE Lite versions for InterBase, Firebird and MySQL. Version 4 introduced full Unicode support, 4.0.3 fixes some issues found in that initial release and subsequent releases, including the "Query was empty"-error in some parts of the MySQL module. There have been numerous improvements to existing tools and the user interface making it even better than before. For more inf...

Pivottable feature(?) in Excel 2003 SP-1
In all earlier versions of Excel, pivottable page, row, & column elements could not be changed...now, in 2003, we find that the text of an item can be over typed, retained, and not returned to it's original form during a refresh. This seems a little odd. Is this a new 'feature' or just a new bug? Either within the Field Options dialog or by typeing directly over the Field Name in a cell, there has been at least one way to change the Field Names permanently at least as far back as Excel 2000. tj "Greg Goodman" wrote: > In all earlier versions of Excel, pivot...

Sheet 1/2 to be available in EXCEL
When using Excel we create many pages. At any time we dont know in which page we are and how many pages are there in total. This problem will be more prolific when the names of pages are all changed from Sheet1, Sheet2 to customised names ---------------- This post is a suggestion for Microsoft, and Microsoft responds to the suggestions with the most votes. To vote for this suggestion, click the "I Agree" button in the message pane. If you do not see the button, follow this link to open the suggestion in the Microsoft Web-based Newsreader and then click "I Agree" in...

Office 2007 Help Tab Not Available For All Users
The new Office 2007 Help tab (the "Get Started" tab replacement) appears to have a problem with multiple users. If I install it under my administrator account, then other accounts, including both admin and user level accounts, do not see the Help tab. It appears that I have to manually install it for each of the users on the computer. Has anyone else seen this problem? Also, does anyone have any ways to workaround this problem? For clarification, I am referring to the Help tab located at http://www.microsoft.com/downloads/details.aspx?familyid=CFB1C570-89A8-44B6-B4...

Zooming features for charts
Has anyone else tried writing a zoom feature for Excel charts? (i.e., dynamically zoom in/out to an Excel chart). I've just finished writing one ... seems to work well for all charts (unless they're 3D): wondered whether there were any off-the-shelf products out there to compare against? Jason - I wrote one a while back for a colleague who couldn't find anything off the shelf. I haven't even looked at it in over two years, either, so I'm not sure I'm up to a competition! I never tried to make it work for 3D charts, but it worked for scatter, line, column, a...

List of available messages does not update
Unless I select a new folder in the left hand pane, the list of messages will not update if I receive new mail in my inbox. I'm running Outlook 2002. Anyone have any ideas? 'OL2002: You Cannot Receive New E-mail Notifications in Environments That Use the Network Address Translation http://support.microsoft.com/default.aspx?scid=kb;en-us;305572&Product=ol2002 --� Milly Staples [MVP - Outlook] Post all replies to the group to keep the discussion intact. Due to the SWEN virus, all mail sent to my personal account will be deleted without reading. After searching google.groups...

Visio 2003 Bug?
I finally figured out that when copy-paste-link from Visio to Word, certain fonts past well and other do not (particularly if one prints to Acrobat PDF). The fonts that do print apparently must have corresponding BOLD, ITALIC, BOLD ITALIC, fonts also installed. Unicode and Dingbat fonts are not liked at all my Visio-Word-Acrobat. I've even tried to trick the three programs by using Font Creator to copy the most used dingbats into a normal font file replacing glyphs A,B,C, etc. -- but these programs are just too smart for me. Has anyone figured out how to copy and paste a simple Visio c...

how do i write a letter on my new letterhead
i have created a new letterhead for our business but cant seem to type any words on it without using text boxes.how do i save it so i can use it and write on it You have to create text boxes. Why don't you want to? -- JoAnn Paules MVP Microsoft [Publisher] "tpppowerlines" <tpppowerlines@discussions.microsoft.com> wrote in message news:D7738168-ADDC-4F24-8546-853AB14FFDCE@microsoft.com... >i have created a new letterhead for our business but cant seem to type any > words on it without using text boxes.how do i save it so i can use it and > write on it ...

write off documents
In Tools-Routines-Sales-Write Off Documents the system currently pulls the default AR account vs. the actual account when doing a writeoff per transaction (ex. in the manual payment apply window when you take a writeoff). When using this feature, the account should be pulled from the document and not per customer and figure out what distribution to use from the transaction (instead of the customer). That way when writing off amounts from a specific transaction, the write-off would be to the same accounts as the original transaction. ---------------- This post is a suggestion for Micr...

Exchange features
Hi everyone. I've never used Exchange server, and I'm in the process of investigating this product. My CEO is concerned that unless we implement a strict email policy, a damaging email can cause a lawsuit. He wants strict control plus less bandwidth without terribly restricting information exchange. The question is, can the Exchange server (2003 or 2007) do the following? 1) Prevent users from using the CC and BCC to send email. 2) Restrict users to send only one email at a time, using the TO field. 3) Option of above two restrictions being controlled through group or user permiss...

New WorkBook With Specified Number of Sheets
Sure would oblige if the proper complete syntax is provided in respect of Creating a New Workbook with a specified number of sheets. Thanx in advance! -- Best Regards, Faraz Faraz A. Qureshi;587147 Wrote: > Sure would oblige if the proper complete syntax is provided in respect > of > Creating a New Workbook with a specified number of sheets. > > Thanx in advance! > -- > Best Regards, > > Faraz From Help: **************************** Creates a new workbook. The new workbook becomes the active workbook. Returns a Workbook object. exp...

How do you make some characters in a cell bold and some not?
I want to make some characters in a single cell bold and some not. How does one format characters differently in the same cell? Select the cell In the formula bar, select the characters that you want to bold On the formatting toolbar, click the Bold button When finished, press the Enter key. tracman wrote: > I want to make some characters in a single cell bold and some not. How does > one format characters differently in the same cell? -- Debra Dalgleish Excel FAQ, Tips & Book List http://www.contextures.com/tiptech.html NOte that Debra's solution works only for cells c...

Invalid high surrogate character (0xDEB6).
I am building an XML document using the standard calls to create elements and attributes. Occasionally I get the following when calling xmlDoc.OuterXml. What's going on and how do I avoid this? System.ArgumentException occurred Message="Invalid high surrogate character (0xDEB6). A high surrogate character must have a value from range (0xD800 - 0xDBFF)." Source="System.Xml" StackTrace: at System.Xml.XmlTextEncoder.Write(String text) at System.Xml.XmlTextWriter.WriteString(String text) at System.Xml.XmlText.WriteTo(XmlWriter w) at Sys...

Web Sevices available in GP 8?
My organization is just beginning the implementation of our AR module in version 8. We are wanting to integrate with our enterprise system, preferably using web services. The help file for eConnect 8 has web services in the architecture and configuration sections, however I haven't be able to locate any other documentation on this ability. Basically the issue is that we plan to skip 9 and move to 10 late this year should it be released. We need to start our integration as soon as possible. If we can somehow begin with web services in 8 and convert later to 10, that would be gr...

Tool Bar Button Image Bug
Hi, I'm using Word in Mac Office 2004. I like to create custom buttons in the toolbar for various little macros I run. Back on my PC (I recently swtiched) Word 2000 has a built in editor for button images. Word 2004 on the Mac lacks this, but does allow you to paste any image onto a toolbar button. However, there seems to be a bug -- the image I paste seems to be reduced to a smaller size than Word's own toolbar buttons. It does this when I try to paste an image file such as a GIF or BMP. Anyone know of a workaround to this? Alternately, anyone have the ability to report this bug t...

trim to first "-" character
Sorry if this is posted again, but I couldn't find my post. How do I trim off everything to the left of the first "-"? The number of spaces are different (to the left of the first "-") so I'm hoping there's a way. There's always a way! (abc) -PLhagdr-hdk (ded ) -GHld-usj (ab) -hgrh-isjb Results would be: PLhagdr-hdk GHld-usj hgrh-isjb On 13 dec, 20:58, Keith <Ke...@discussions.microsoft.com> wrote: > Sorry if this is posted again, but I couldn't find my post. > > How do I trim off everything to the left of the f...

Advantages to Table Feature in Excel 2007?
I have a question that I have never had to deal with as an Office 2000-2007 Trainer at my company. One of our departments needs to know how to use the various Excel 2007 features. One thing on their list is the Excel 2007 Table feature. 1) It seems like an interesting option that automates many of Excel's features. I typically do not use the Table feature and teach how to do the various options without using the Table option. This upcoming training has me thinking. Should I just teach the Table option and not worry about the students understanding many of the underlining ...