What could cause XML files to not be completely written?

We've had a month long headache with one of our functions in the CMS we've 
built.

Whenever a page is updated in our CMS, we do a recursive call to the DB to 
get all the pages (and child pages, and child child pages, etc.) and then 
create an XML file, and save that to the filesystem. The front end then uses 
that XML file along with some XSL to create the site menus, site maps, etc.

We're having a problem where 99 out of 100 times, this works fine. But that 
one time, we'll get an incomplete (and therefore invalid) XML file written 
to the filesystem. No error. Nothing. Just a bad file.

What could be the possible causes of this? I've included the function that 
writes the XML at the bottom of this message if that helps.

At this point, we're looking into two solutions which just seem a bit hacky:

1) write the file, open the file, validate it, if invalid, write again, 
repeat a few times and then report an error if you have to.

2) forget the filesystem. Write the XML file as a VARCHAR back into a DB 
table and just grab it from there.

Any other thoughts? Things to look out for? Solutions? Theories?

Thanks!

------------------------------------------------------------------

Imports System.IO
Public Class menuWriter
Shared fs As System.IO.FileStream
'shared objXMLWriter As system.xml.XmlTextWriter

Shared Sub createXMLfileWithXmlWriter(ByVal parentNode As Integer, ByVal 
intLevel As Integer, ByVal rowCount As Integer, ByVal createParentNode As 
Boolean, ByVal siteID As Integer)
Try
' Create the parent node
'dim objXMLWriter As system.xml.XmlTextWriter
' xml writing adapted from
' http://www.developerfusion.com/show/2646/
If 
System.IO.Directory.Exists(System.Web.HttpContext.Current.Server.MapPath("/mjb05/xml/" 
& siteID)) Then
'directory exists. Do nothing
Else
'directory doesn't exist. Creat directory
System.IO.Directory.CreateDirectory(System.Web.HttpContext.Current.Server.MapPath("/mjb05/xml/" 
& siteID))
End If

Dim objXMLWriter As System.Xml.XmlTextWriter
Dim path As String = 
System.Web.HttpContext.Current.Server.MapPath("/mjb05/xml/" & siteID & 
"/siteMenu.xml").ToString
fs = New FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None)
SyncLock fs
objXMLWriter = New System.xml.XmlTextWriter(fs, 
System.Text.Encoding.Default)
objXMLWriter.Formatting = objXMLWriter.Formatting.Indented
objXMLWriter.Indentation = 3
objXMLWriter.WriteStartDocument()
objXMLWriter.WriteComment("Created on " & Now())
objXMLWriter.WriteStartElement("menuItems")
WriteXmlData(objXMLWriter, parentNode, siteID)
objXMLWriter.WriteEndElement() 'close menuItems node
objXMLWriter.WriteEndDocument() 'close docment
objXMLWriter.Flush()
objXMLWriter.Close()
End SyncLock
Catch ex As Exception
System.Web.HttpContext.Current.Response.Write("Error writing XML file<br 
/>")
System.Web.HttpContext.Current.Response.Write(ex.Message.ToString)
End Try
End Sub

Shared Sub WriteXmlData(ByVal objXMLWriter As System.Xml.XmlTextWriter, 
ByVal parentNode As Integer, ByVal siteID As Integer, Optional ByVal 
intLevel As Int32 = 0)
'retrieve all children of parentNode
' grab the data from the db
Dim DS As New DataSet
Dim strConnect As String
strConnect = 
System.Configuration.ConfigurationSettings.AppSettings("DBConn")
Dim strChk As String
strChk = "SELECT pageID, siteID, parentID, linkText, browserTitle, 
contentTitle, hidden, sort, customContent, customContentVariables, 
menuCategory, isCategory, homePage " & _
"FROM WeSiteMenus " & _
"WHERE ParentID = " & parentNode & " AND siteID = " & siteID & " " & _
"ORDER BY homePage DESC, sort ASC"
Dim objConnect As New System.Data.OleDb.OleDbConnection(strConnect)
objConnect.Open()
Dim objCommand As New System.Data.OleDb.OleDbCommand(strChk, objConnect)
Dim objOleDbAdapter As New System.Data.OleDb.OleDbDataAdapter(strChk, 
strConnect)
objOleDbAdapter.Fill(DS, "webPages")

' Walk through results whether it's a list or a single contact.
' based off of this tutorial:
' http://www.wwwcoder.com/main/parentid/154/site/899/68/default.aspx
Dim rowCount As Int32 = 0
While rowCount < DS.Tables(0).Rows.Count 'ie, if there IS data, then...
objXMLWriter.WriteStartElement("menuItem")

objXMLWriter.WriteElementString("pageID", 
Trim(DS.Tables(0).Rows(rowCount)("pageID").ToString))
objXMLWriter.WriteElementString("parentID", 
Trim(DS.Tables(0).Rows(rowCount)("parentID").ToString))
'linkURL...need to actuall add this to the DB!
objXMLWriter.WriteElementString("linkURL", "")
objXMLWriter.WriteElementString("linkText", 
Trim(DS.Tables(0).Rows(rowCount)("linkText").ToString))
objXMLWriter.WriteElementString("browserTitle", 
Trim(DS.Tables(0).Rows(rowCount)("browserTitle").ToString))
objXMLWriter.WriteElementString("contentTitle", 
Trim(DS.Tables(0).Rows(rowCount)("contentTitle").ToString))
objXMLWriter.WriteElementString("hidden", 
Trim(DS.Tables(0).Rows(rowCount)("hidden").ToString))
objXMLWriter.WriteElementString("customContent", 
Trim(DS.Tables(0).Rows(rowCount)("customContent").ToString))
objXMLWriter.WriteElementString("customContentVariables", 
Trim(DS.Tables(0).Rows(rowCount)("customContentVariables").ToString))
objXMLWriter.WriteElementString("menuCategory", 
Trim(DS.Tables(0).Rows(rowCount)("menuCategory").ToString))
objXMLWriter.WriteElementString("isCategory", 
Trim(DS.Tables(0).Rows(rowCount)("isCategory").ToString))
objXMLWriter.WriteElementString("level", intLevel.ToString)
'now call the subroutine we're in to see if this value has
'any children and increase the indent, and so on...
WriteXmlData(objXMLWriter, 
Convert.ToInt32(DS.Tables(0).Rows(rowCount).Item(0)), siteID, intLevel + 1)
rowCount = rowCount + 1
objXMLWriter.WriteEndElement() 'close menuItem node
End While
' clean up
objConnect.Close()
objOleDbAdapter.Dispose()
objCommand.Dispose()
End Sub
End Class 


0
5/11/2006 6:46:11 PM
dotnet.xml 7266 articles. 0 followers. Follow

3 Replies
493 Views

Similar Articles

[PageSpeed] 42

darrel wrote:
> We've had a month long headache with one of our functions in the CMS we've 
> built.
> 
> Whenever a page is updated in our CMS, we do a recursive call to the DB to 
> get all the pages (and child pages, and child child pages, etc.) and then 
> create an XML file, and save that to the filesystem. The front end then uses 
> that XML file along with some XSL to create the site menus, site maps, etc.
> 
> We're having a problem where 99 out of 100 times, this works fine. But that 
> one time, we'll get an incomplete (and therefore invalid) XML file written 
> to the filesystem. No error. Nothing. Just a bad file.

You mean the file just ends in mid-element or something?
The only obvious things that come to mind are a buggy serializer
or a timing problem between the DB and the OS or disk subsystem.

> What could be the possible causes of this? I've included the function that 
> writes the XML at the bottom of this message if that helps.
> 
> At this point, we're looking into two solutions which just seem a bit hacky:
> 
> 1) write the file, open the file, validate it, if invalid, write again, 
> repeat a few times and then report an error if you have to.
> 
> 2) forget the filesystem. Write the XML file as a VARCHAR back into a DB 
> table and just grab it from there.

Easier would be to run an external standalone validator (ie unconnected
to and independent of any other part of your system) which will pass
control onwards if the file is OK, but otherwise terminate the process
and restart it at the file export step.

///Peter
-- 
XML FAQ: http://xml.silmaril.ie/
0
Peter
5/11/2006 7:55:24 PM
> You mean the file just ends in mid-element or something?

Yep. It'll just cut of mid-element. Randomly anywhere in the document.

> The only obvious things that come to mind are a buggy serializer

I'm not sure if I'd know what to look for regarding that. What would that 
actually be?

> or a timing problem between the DB and the OS or disk subsystem.

Well, I'm grabbing all the data from the DB at once, and then writing it to 
the XML file in one swoop, so that probably isn't it. No doubt it could be 
the OS or disk subsystem, though. NO idea how to even begin to figure out 
what specifically to look at in that context, though.

> Easier would be to run an external standalone validator (ie unconnected
> to and independent of any other part of your system) which will pass
> control onwards if the file is OK, but otherwise terminate the process
> and restart it at the file export step.

Whoa. So, I'd be passing something to another app and back again? I'm not 
sure if that sounds easier to me ;o)

What kind of external validator are we talking about? Just another web app 
that I'd whip up? What would be the advantage of externalizing the validator 
(vs just validating what was written in the app itself?)

-Darrel 


0
5/11/2006 8:05:27 PM
darrel wrote:
> What kind of external validator are we talking about? Just another web app 
> that I'd whip up? What would be the advantage of externalizing the validator 
> (vs just validating what was written in the app itself?)

Any of the standalone parsers will do: it doesn't actually have to
validate if all you want is to test the file's completeness. If we
assume that the file is either complete (and correct) or incomplete
(and that no other mid-way corruption takes place) then a simple
well-formedness check will show if the file is complete or not.

My reason for suggesting an external process was simply to guarantee
that the parse was not being affected by anything else that might have
gone on inside your app.

///Peter
-- 
XML FAQ: http://xml.silmaril.ie/
0
Peter
5/13/2006 12:23:58 AM
Reply:

Similar Artilces:

Print file path
I want the file path to be printed on every excel wooksheet I print. Is there a way to do this on new and existing worksheets without having to go into Headers and Footers each time? Thanks, Esther See http://www.mcgimpsey.com/excel/fullnameinfooter2.html In article <13B70E9B-37DE-4A55-9734-8D465E635DA6@microsoft.com>, "EstherJ" <EstherJ@discussions.microsoft.com> wrote: > I want the file path to be printed on every excel wooksheet I print. Is > there a way to do this on new and existing worksheets without having to go > into Headers and Footers ...

Read Text File into Excel Using VBA
Hi All, I'm a new VBA programmer. I know how to pull an entire text file into an Excel Spreadsheet, but I only want specific information from the text file not the entire text file. What I have is about 25 text files stored in a folder, let's say C:\test. Each file is named by a property address as follows: 209 MAIN ST.txt 213 MAIN ST.txt 111 ELM ST.txt 2356 WOOD AVE.txt On the 11th row of each file is as follows: Property Address:209 MAIN ST On the 31st row of each file is as follows: Total Value:30500 What I would like to do is read each file located in the "C:\test folder...

Outlook 2003 cannot find my files
I am running Windows XP Professional Tablet. I upgraded to Office 2003, and now I cannot see my contacts and my calendar is empty. I do not know what to do now, as I cannot run the earlier version of Outlook. I can find the pst files, but I cannot figure out how to import them into this new version. In addition, I cannot see the local email files that I have stored in my directory. Help! Sincerely, GVG Don't try to import - use File->open->outlook data file and browse to the location of your .pst file. --� Milly Staples [MVP - Outlook] Post all replies to the group to ke...

File view doesn't list an expected source file
Hi, File view isn't showing a file it should show. One of the last steps needed to add Help support to an existing project is to compile MyProject.hpj. To do this, msdn says that I should go to File view, open the source files list, highlight MyProj.hpj, and then select "Compile MyProject.hpj" in the Build menu. MyProject.hpj is not in the Source files list, thogh. Can someone tell me why it isn't there? The file exists in my project's MyProject\hlp directory as a result of copying the sample project's hlp directory to my project, and renaming the files so th...

File opening problem over network
I have what seems to be a strange excel file opening issue that I can't pinpoint and I would greatly appreciate any ideas to try for this problem. We have a network computer running windows 98 SE. On this computer there are many .xls files stored for users to access. Often when opening the files users get an error that reads: Microsoft Excel cannot open or save any more documents because there is not enough available memory or disk pace. -To make more memory available, close workbooks or programs you no longer need. -To free disk space, delete files you no longer need from the disk yo...

Dialog members cause Debug Assertions?
I have this problem that occurs with several dialog member variables. I have this custom dialog that I run as a modal dialog, and copy some data from it when it returns IDOK. This goes something like this: CPalDlg dlg; if (dlg.DoModal() == IDOK) { m_pal = dlg.m_spinPal.GetPos(); } Where CPalDlg is my custom dialog class, m_pal is an int member variable, and m_spinPal is a CSpinButton member variable of the dialog. I can't spot any mistakes, but when I run it, I get a "Debug Assertion Failed" message. Going through the call stack, it turns out the "m_pal = dlg.m...

Increasing recently opened files list
Does anyone know how to increase number of files shown in recently opened files list? Mine shows four, and I'd like eight or ten, the same as I get in all my other MS applications. Any auggestions?? Thanks. tabboo wrote: > Does anyone know how to increase number of files shown in > recently opened files list? Mine shows four, and I'd like > eight or ten, the same as I get in all my other MS > applications. Any auggestions?? Thanks. =================================== What is the name and version of the program you are referring to? -- John Inzer return e-mail disabled...

Tracking Attachment File Names
Why is Outlook 2003 so reluctant to expose details of attachments to the user. I would like to be able to search for, view, and print a report that targets the file names of attachments. The attachment file name only appears on a printed message if the message is formatted as plain text. Even then, the file name is only fully displayed if it is not too long to fit in the space allocated. As for being able to search for a file name, to see which message(s) contained that attachment, that just seems too hard. Utilities that strip attachments from messages and then insert a text string in th...

Deleting Multiple PST Files
I now have 3 sets of personal folders in Outlook 2002. I need to delete 2 of them so I can eliminate the confusion when saving files. How can I delete the additional personal files. I tried deleting Outlook and reinstalling but it automatically kept everything. Dear Brian, have a look on this site: http://office.microsoft.com/en-us/assistance/HA011170471033.aspx may it helps. -- Oliver Vukovics Public OutLook: Share Outlook without Exchange Public SyncTool: Outlook PST Synchronization www.outlookstore.com Brian wrote: > I now have 3 sets of personal folders in Outlook 2002. I ...

Max size of OST files
Hi all, I'm making this post on behalf of a friend. They have a single Exchange2000 server. Not sure if it's SBS or Exchange2000 Std. There are approx 20 mailboxes on this server. The users all use Outlook 2003 (SP3) as their mail client and Outlook's account type is Exchange and not POP3 or IMAP. One of the users mailboxes is approx 6Gb in size. What they would like to do is use offline mode to create a .ost file of this persons mailbox for his laptop however the file limit on ost files appears to be 2Gb. We did find a MS k/b article that explained how to create the requi...

Problem opening files in excel with a specific account
Hi. We have problmes opening a couple of files in excel, without any specific error message. The application just "hangs". When we log on to WXP as another user, the files work fine. We have re-installed excel, deleted and recreated the user profile which has problems, but still no luck. Anybody got a clue why this is happening? The computer excel is running on is a DELL Optiplex Pentium HyperThreading machine. Thansk in advance. Sometimes cleaning up the windows Temp folder will help. If that doesn't help, sometimes the file that contains the toolbar customizations...

Invoice Number should be the name of file on pdf
When printing an invoice to a pdf format, or emailing the invoice, the system should automatically prompt you to name it as the invoice number, not "SOP Blank Document.pdf". ---------------- This post is a suggestion for Microsoft, and Microsoft responds to the suggestions with the most votes. To vote for this suggestion, click the "I Agree" button in the message pane. If you do not see the button, follow this link to open the suggestion in the Microsoft Web-based Newsreader and then click "I Agree" in the message pane. http://www.microsoft.com/Businesss...

is there a "File name" function in ms Excel (2002)
Hi Anyone know of any msExcel function that will give the name of the current file? To be honest I want if for printout purposes because I am trying to avoid the custom header etc (which seems difficult/impossible to format) Shiphen Shiperton Henethe One way =LEFT(MID(CELL("filename",A1),FIND("[",CELL("filename",A1))+1,255),FIND(".", MID(CELL("filename",A1),FIND("[",CELL("filename",A1))+1,255))+3) can be put in any cell if you want the full path =SUBSTITUTE(LEFT(CELL("filename",A1),FIND("]",...

want to download csv file from website and need to open in Excel.
currently the file is opening in html, then i save it as a text file, then manually open Excel and finally open the text file and then the file opens in csv format. this used to happen behind the scenes then I received a new computer. i am sure there must be a switch that needs to be toggled but at am at a loss as to what that swithc might be. any ideas? from: http://support.microsoft.com/kb/q162059/ 1. Open My Computer. 2. On the Tools menu (or the View menu), click Folder Options (or click Options). 3. Click the File Types tab. 4. In the Registered file types list, click the specific...

File Attachment Help !
I have created table to keep a track of attached file info. Fields are : CustID, FileName,FileLocation In my main form I have created “Attached File” button – (to attach file – should attach more than one) and “View Attachment” - (to view all attached file) button. I have created two list box – one to view the list of attached file (Current attachment). And other to view the list which is already there (previously attached) I need to allow users to upload attachments (docs, jpgs, tifs, etc.) to a network drive. And View the attachment when click on “View Attached File” but...

Recover Outlook 2002 Archive file from Norton Protected Recycle Bin
Deleted Outlook 2002 backup/archive file by mistake. Found it in Norton Protected Recycle Bin. Made copies then tried to restore one copy from Nortons and one copy through Outlook import, both attempts failed with same results. Outlook does not recongonize file as personal folders file. Any suggestions. Assuming read only hasnt been set, Try running the repair tool, scanpst.exe on the pst. You may have to run it several times. Then in OL file/open, the pst "Larry" <onsitecs@fuse.net> wrote in message news:188ce01c44af4$0b2bc4f0$a501280a@phx.gbl... > Deleted Outlook 200...

Open XLS file without it's number format using VBA?
An XLS file has been saved with it's numbers formatted to a specified number of decimal places. Using VBA how do you read in that XLS file with the numbers unformatted and with the original number of decimal places? Workbooks.Open (filename:="myFileName.xls", ????) Not sure what you mean by "read in". The values are still in the cells. If you open the workbook, then transfer the cell values from the worksheet to VBA, you'll get the entire number, not the version that is rounded for viewing purposes. On Wed, 06 Oct 2004 22:55:55 GMT, Dennis@NoSpam.com wrote: >...

Open File Path not as expected
Version: 2008 Operating System: Mac OS X 10.6 (Snow Leopard) Processor: Intel When I open a file via the OPEN dialog, I expect Word to open that same folder (subdirectory) the next time I OPEN a file. But it doesn't. The OPEN dialog opens to a folder up a level or two from the previously defined target. What up? You're expecting the Windows Word behaviour. Mac Word doesn't do that. Many of us wish it did, but Mac OS has a different concept of "Current Folder" from Windows, and apparently it would be against the rules for Microsoft to replicate the Windows beh...

Locking excel file
Hi how do you lock an excel file so that other people using the computer will not be able to open? dolin, http://youngindonesiangirls.googlepages.com File>Save As>Tools>General Options.......set a password to open Gord Dibben MS Excel MVP On Mon, 14 Jul 2008 15:27:30 -0700 (PDT), Dolin <ceweindo@gmail.com> wrote: >Hi how do you lock an excel file so that other people using the >computer will not be able to open? > >dolin, >http://youngindonesiangirls.googlepages.com hey thanks gord, ive been searching this for quite some time. On Jul 15, 3:53=A0am, Gord ...

Output Query as an Excel file
I am trying to output a query as an excel file and save it to a specific location. Here is my code. DoCmd.OutputTo acOutputQuery, "Qry:SKU_ALL_2", acFormatXLS, _ "C:\RH DOC\LCM_PROJECT\LCM_QRY" & "\" & "LCM_QRY" & "_" & "PERIOD" & "_" & ([Forms]![LCM_DATA]![PER]) & ".xls", False I get a run time error '2306' that states there too many rows to output, based on the limitation specified by the output format or by Microsoft Access. There are 36,464 records. When I take...

Indicator for tasks extending beyond project completion date
1. Our project has a targetted completion of June 30, 2010. 2. For visual purposes, I created a milestone 'Project complete - target June 30, 2010' which has a deadline of June 30 and a 'Must Finish On' June 30 constraint (knowing that this is an inflexible constraint) so that I can visualize the milestone and deadline on the same June 30 date. 3. How do I have indicators appear against tasks that end beyond the targeted project end date? Thanks Hi, Link the task to the projected end and they will show negative slack (whihc, through a simple formula in a ...

I'm having problems inserting text files from notepad
When I try to insert a text file created in notepad, which contains characters such as £, $ or letters with symbols over them, into a text box in Publisher 2003 it comes up with a File Conversion window, and the options to choose the Text encoding option. This didnt happen in our older version of Publisher, on our old PCs. We have new PC running Windows XP and Office Professional 2003 and I want to be able to insert the files without this window appearing. I have checked the registry and CharacterSet value is set to ANSI. Any suggestions on what is happening and how I can correct...

.tmp files #3
Excel 2000 is leaving the tmp file in the folder when any spreadsheet is opened and closed. This is confusing to the users and is wasting network drive space. I found that it should be renaming and/or deleting these files. Is this a feature or is there some setting I am missing. "Mike Scott" <anonymous@discussions.microsoft.com> wrote... >Excel 2000 is leaving the tmp file in the folder when any >spreadsheet is opened and closed. This is confusing to >the users and is wasting network drive space. >I found that it should be renaming and/or deleting these >fi...

File Dialog Explorer style + manipulate existing controls #3
First, thx for your replies on my first question. I ave another question: in my application I have an online-language-switch option, where the user can switch between the languages German, English and French. Can i manipulate the buttons and labels of the modern file-open-dialog at runtime? I don't want to add new controls only manipulate the existing one (setting correct language texts). Is this possible with the file dialog which uses explorer style? thx ...

Creating XML Master/detail in one line per record.
I'm using VS (in ASP.NET VB) for a few months and I'm creating my first site. But I'm not that familiar with XML. Only the basic stuff. What I'm want to create is a XML file that will be used as a gridview source. The file should contain master/detail data with the following requirement : The master field values (M) and the corresponding detail field values (D) of a DB record, should be displayed in the same gridview row as if they were fields from a single table. That is like: M1|M2|M3|D1|D2|D3|D4. How can I create an XML file like that? I want it to be an XML file, so...