Script to delete millions of files

We just inherited a server that has a folder which contains millions
of old log files that need to be cleaned up.
Using the GUI to even view the folder, let alone to do anything in it
obviously just hoses the server with so many file handles.

We tried deleting files from the folder using a command prompt but ran
into similar issues.

I'm thinking maybe we can write a script that uses the folder's file
collection and read in maybe 500 file names at a time into an array.
Then we could then loop through the array and delete the files one at
a time, then read in another 500 and so on. This script could run over
a weekend and maybe finish by monday morning.

Problem is I've been focused on learning CF and SQL for the past year
and my VBS skills are a bit rusty. So I thought I would just see if
anyone has something close to this already that I could use as a
starting point.

Thanks.
0
Mickey
12/8/2009 3:40:58 PM
scripting.vbscript 564 articles. 0 followers. Follow

6 Replies
815 Views

Similar Articles

[PageSpeed] 28

 You can try looping through the Files collection
and calling File.Delete on each one. you don't
need the intermediate step of an array. But....
maybe it'd be easier to just format and re-install.

> We just inherited a server that has a folder which contains millions
> of old log files that need to be cleaned up.
> Using the GUI to even view the folder, let alone to do anything in it
> obviously just hoses the server with so many file handles.
>
> We tried deleting files from the folder using a command prompt but ran
> into similar issues.
>
> I'm thinking maybe we can write a script that uses the folder's file
> collection and read in maybe 500 file names at a time into an array.
> Then we could then loop through the array and delete the files one at
> a time, then read in another 500 and so on. This script could run over
> a weekend and maybe finish by monday morning.
>
> Problem is I've been focused on learning CF and SQL for the past year
> and my VBS skills are a bit rusty. So I thought I would just see if
> anyone has something close to this already that I could use as a
> starting point.
>
> Thanks.


0
mayayana
12/8/2009 3:49:51 PM
On Dec 8, 10:49=A0am, "mayayana" <mayaXXy...@rcXXn.com> wrote:
> =A0You can try looping through the Files collection
> and calling File.Delete on each one. you don't
> need the intermediate step of an array. But....
> maybe it'd be easier to just format and re-install.
>

Yeah I wish that was an option but this is a production web server and
it needs to stay up and running.
I guess I was just trying to find a way to limit the stress on the
master file table by deleting files in chunks. Not sure if that really
makes a difference or not. I thought maybe if I only open the files
collection long enough to grab 500 items and then close it, I might
leave more memory free to process the deletions. It's all just
theory... and very possibly overkill. I just need to avoid locking up
the server.
0
Mickey
12/8/2009 4:10:41 PM
"Mickey" <xyxjunkmale@gmail.com> wrote in message 
news:163687c0-45f6-4ca6-b765-7c3219aaa175@9g2000yqa.googlegroups.com...
> We just inherited a server that has a folder which contains millions
> of old log files that need to be cleaned up.
> Using the GUI to even view the folder, let alone to do anything in it
> obviously just hoses the server with so many file handles.
>
> We tried deleting files from the folder using a command prompt but ran
> into similar issues.

Could you be a little more specific and explain what exactly you mean with 
"hosing the server", in particular when using console commands. Also - what 
console command did you use? 


0
Pegasus
12/8/2009 4:23:52 PM
On Dec 8, 11:23=A0am, "Pegasus [MVP]" <n...@microsoft.com> wrote:
> "Mickey" <xyxjunkm...@gmail.com> wrote in message
>
> Could you be a little more specific and explain what exactly you mean wit=
h
> "hosing the server", in particular when using console commands. Also - wh=
at
> console command did you use?

I am repeating what someone else told me... I have not tried this
myself but was just trying to offer them a solution based on what I
was told... that using the GUI (Windows Explorer) took an impossibly
long time to enumerate the files (longer than an hour) and after any
attempted action the process started over and eventually locked up the
server requiring a reboot.

It's moot now. Someone else is cleaning up the files though I wasn't
told how. But it seems to be working.
Thanks anyway.
0
Mickey
12/8/2009 5:29:57 PM
Mickey wrote:
On Dec 8, 11:23 am, "Pegasus [MVP]" <n...@microsoft.com> wrote:
> "Mickey" <xyxjunkm...@gmail.com> wrote in message
>
> Could you be a little more specific and explain what exactly you mean with
> "hosing the server", in particular when using console commands. Also -
what
> console command did you use?

I am repeating what someone else told me... I have not tried this
myself but was just trying to offer them a solution based on what I
was told... that using the GUI (Windows Explorer) took an impossibly
long time to enumerate the files (longer than an hour) and after any
attempted action the process started over and eventually locked up the
server requiring a reboot.

It's moot now. Someone else is cleaning up the files though I wasn't
told how. But it seems to be working.
Thanks anyway.
************

If they did it any way other than opening a command prompt and typing
DEL "path_to_folder\*.log" then too much effort went into it.

Although the deed is done now, if the logs needed to be retained, I was
going suggest moving the logs into folders or zip files sorted by date.

-- 
Todd Vargo
(Post questions to group only. Remove "z" to email personal messages)

0
Todd
12/8/2009 9:43:31 PM
Hi Todd,

> It's moot now. Someone else is cleaning up the files though I wasn't
> told how. But it seems to be working.
> Thanks anyway.
> ************
>
> If they did it any way other than opening a command prompt and typing
> DEL "path_to_folder\*.log" then too much effort went into it.

I have had a server where I tried to do something like that (also with a routine thet kept 
generating log files every hour without me knowing about it) and kept getting errors about not 
having enough ....., don't remember what it was, might have been stack or heap memory. When you are 
talking about millions of files you will be running into strange limitations.

> Although the deed is done now, if the logs needed to be retained, I was
> going suggest moving the logs into folders or zip files sorted by date.

Might have run into the same problem. I'm not sure a standard ZIP file can handle that many 
filenames or whether the ZIP program can handle the number of files.
In my case even braking it up into a bunch of Del a*.* Del b*.* was not enough, I had to break it up 
into Del aa*.* Del ab*.* and so on. Made a batch file that did this for me and it ran over a day.

Bonno Bloksma


0
Bonno
12/9/2009 6:48:21 AM
Reply:

Similar Artilces:

Outlook 2003 and exchange server. I want all downloaded to pst, only 1 calendar in pst, and sent items to pst file
Hi folks, I was just upgraded at work from Outlook 2000 to Outlook 2003. I used to download all mail from the server to a local pst file (which I have done again via a rule). Is this the best way. I think back in 2000 I just said deliver all mail to the local 'inbox' in the pst file. I never used the exchange calendar, and scheduled everything in my personal pst file. Now, items are scheduled in the server, but I want the old behavior back. All my 'sent' mail was stored in my local pst file. Now it's being stored on the server. I want the old behavior back. How do ...

File Attachment Problem
I have got the file attachment problem since we implemented the dynamics CRM 4.0. So i will be glad if you advise me a way to solve this problem. sincerally ...

Reject script in pass through san script not working
I have a scenario that i had to check for dependencies of divisions(Human Resources) before their deletion and decide whether to let deletion proceed or not, i had done this for many entities like department(Dynamics) in my development dictionary, i registered a focus trigger before delete button of department form and checked dependencies for it on triggered script and allow or prevent deletion successfully. Now for division that is in the human resources dictionary i registered a cross-dictionary focus trigger to run before delete buttons change event as result = Trigger_RegisterFocus...

why does deleted email have a line thru it
I have 2 email accounts. One works fine. On the other; when I delete an email I have a line thru it and the email stays in the inbox and cannot be removed. Help. Did you try and hit Send\Receive when the line is through the message. Does it delete then -- Peter Please Reply to Newsgroup for the benefit of others Requests for assistance by email can not and will not be acknowledged. "Michael" <Michael@discussions.microsoft.com> wrote in message news:AFDC8DB7-12D5-47C3-AA1E-74F72CF0258D@microsoft.com... >I have 2 email accounts. One works fine. On t...

Sort Column Script
I need to have a script in Excel that would allow the sorting of columns from column "D" to the column before the last column with data. I do not want the last column sorted. The criteria for the sort is in row 1 (Name of an employee). Additional columns will be inserted between column "D" and the last column so they need to be sorted. I hope this is clear. Can someone please assist me? Thank you! Bernie Bernie How can someone named Bernie NOT know how to do this? ;-) Sub BernieSort() Range(Range(Range("D1"), Range("D1").End(xlDown)), _ ...

No Log Files can be Truncated-Exchange 2003 Backup
Hi all, I recently did a full backup of my A and B information Stores in which A o went fine and truncated all the logfiles but on the B information store it said in the event viewer that "No log files can be truncated and all the log files are there. Any ideas what is going on and what can be done. -Aman On Feb 5, 4:04 pm, Andy David {MVP} <ada...@pleasekeepinngcheesebucket.com> wrote: > On 5 Feb 2007 15:51:25 -0800, "Barundi" <pan...@gmail.com> wrote: > > >Hi all, > > >I recently did a full backup of my A and B information Stores in which ...

Deleting Records in Access
Can anyone tell me a quicker way to delete multiple records in Access? Currently, I am clicking/highlighting/scrolling through multiple rows in my table to delete them. I have around 200,000 to delete. Thanks if you can help!! Bonnie if the records that you are deleting has an identification like:>1000, BL001,BL001 DoCmd.RunSQL "DELETE Table.* FROM Table WHERE [Identfication] -- Lorenzo Díaz Cad Technician "Bonnie" wrote: > Can anyone tell me a quicker way to delete multiple records in Access? > > Currently, I am clicking/highlighting/scro...

C00D10D1 Error
I am on Windows Media Player 10. I cannot play one specific .wav file. The error that comes up is as follows: "You've encountered error message C00D10D1 while using Windows Media Player. The following information might help you troubleshoot the issue. Codec is missing Windows Media Player cannot play the file (or cannot play either the audio or video portion of the file) because the Philips CELP (120) codec is not installed on your computer." I have looked for this specific codec, and so far have not been able to find anything. I cannot update my codecs throug...

can not find "hcw" file
When I migrate a C++ MFC project with ActiveX stuff , encounter an problem like "can not find hcw file.", I found some solutions from MSDN forum, but those are useless.. I'm using Vista 32 and 2005Vs Studio @the momemt. any suggentions? plz.. On Thu, 26 Apr 2007 01:55:30 +1000, "Ramon Chen" <Ramon@live.com.au> wrote: >When I migrate a C++ MFC project with ActiveX stuff , encounter an problem >like "can not find hcw file.", hcw = Help Compiler Workshop or something like that. It creates an old style .HLP file. If you do not care about tha...

Backup .pst files #3
I just bought a new computer and reloaded outlook. I backed up the .pst file and showed 374MB. When I applied the restore of the 2000 e mails backed up only 600 appeared. The dates for the e mail are 2002 and 2003, no 2004. Near the end of 2003 I was using XP Pro and the program became corrupt. I bought XP Home edition. Why is back up not backing up or coping. Please help. How did you backup the pst, the OL backup addin? Have you tried opening the pst in OL, rather than import. If on opening pst, mail is missing, then backup went wrong. Assuming old sys is still available copy pst to cd...

Missing file
Hi there, I am hoping someone out there can help me. I have just migrated from OE to Outlook 2002(with Nortons Internet Security.) When Outlook loads I keep getting a message saying "C:\Windows\System32\fxsext32.dll" is missing or corrupt etc. I have tried using the Outlook Repair Facility to no avail. The error only comes up once after Outlook is started; eitherwhen it finds a spam file for the first time or when its sending an email for the first time. many thanks Ric ...

--------Open Excel File
When double-click on an xls file, Excel 2002 got opened but the file did not. So I have to go to File - Open menu to find the file and open it. Any solution? Thanks. Hi in Excel goto 'Tools - Options - General' and uncheck 'Ignore other applications' -- Regards Frank Kabel Frankfurt, Germany "Ian" <Ian@discussions.microsoft.com> schrieb im Newsbeitrag news:5C123077-3761-4A0D-868E-5CB1087D7A9A@microsoft.com... > When double-click on an xls file, Excel 2002 got opened but the file did not. > So I have to go to File - Open menu to find the file and open ...

Delete two columns with formula
I need to delete 2 columns (with formula) and only display the last two columns. When I delete the first 2 columns, the last two columns display #REF!. I know I can copy the info I need to a new sheet and paste special the value. Is there a way to fix this on the current sheet I am working on. Thanks. "ED" <ED@discussions.microsoft.com> wrote in message news:1944C248-624E-4D95-AD8E-86412991A77F@microsoft.com... >I need to delete 2 columns (with formula) and only display the last two > columns. > > When I delete the first 2 columns, the last two columns di...

How to make PST file default?
I've created another PST file in Outlook 2002. How do I make this one the default for mail? I have two PST files open. Thanks, Brett - Go to Tools > E-mail Accounts > View or change existing e-mail accounts - Select the PST you just added in the "Deliver New e-mail to the following location" dropdown (it will most likely appear just below the current default Personal Folders file) > Finish - Exit and Restart Outlook - You can now close the previous default PST (R-click on that pst folder while in Folder View and select "Close <foldername> Folders")....

Records are not deleted in the database
Hi everybody, today we noticed when we are deleting records in the web client, they stay in the database with the value 2 in the DeletionStateCode Column. I guess normally this is the markup for some sort of job or service to clean these records, but apparently this never happens. We have deployed Rollup Package 2 which should include a hotfix for this problem, but no luck. Can anyone point us in the right direction or is this a (huge) bug? Thanks Christian On Feb 3, 5:51=A0pm, CGuntsche <CGunts...@discussions.microsoft.com> wrote: > Hi everybody, > > today we noticed whe...

Insert hyperlink to .dot file to create a new document
Good morning, I'm trying to insert a hyperlink to a .dot file - and that works fine, except that I want to create a new word doc - not open the .dot file. What am I doine wrong Kind Regards Pysse Maybe you could put a button from the forms toolbar on the worksheet and assign it a macro to open a new word document based on your template: Option Explicit Sub testme01() Dim oWord As Object Dim myWordTemplate As String Dim testStr As String myWordTemplate = "c:\msoffice\templates\1033\Elegant Fax.dot" testStr = "" On Error Resume N...

files in c:\windows\installer have invalid digital signature
This is the advanced question about files in c:\windows\installer:I&nbsp; have noticed that some of msi-files located there have invalid digital signatureand some have valid one. What is it caused by? If these files are created by WindowsInstaller why some have correct signature and some not? Is there a way to repairthe installer?The second question is about files in the same directory created by InstallShield installerlike ARPPRODUCTICON and shorcuts. If i look the properties of these files they show somethingodd: there is two line menu on the top of &#119;indow. Again is there a way ...

Mailbox Deletion
We're a W2K3/Ex03 Environment. We also need to adhere to SarbOx compliancy. For administrative/SOX issues we'd like to be able to delete a user account, but perseve the users mailbox. Is this possible? Or do we need to disable the account only in order to keep the mailbox available? the practical thing to do probably, is to make a full copy of the mailbox, including items in the dumpster, before deleting the AD account. You can, of course, set Exchange to not purge mailboxes for a long time, but you'd need to reconnect the mailbox to another user account in order to access the...

How do I re-install the Contacts file?
Due to a corrupt Windows file I had to re-install Windows. Unfortunately this means that the new Outlook Contacts files, and files I had set up in "My Shortcuts", do not show up in Outlook. are emptyHow can I move files from the original Outlook (specifically the information in "Contacts" and files I had set up in "My Shortcuts"? files from the old Outlook into the current Outlook? Thanks, Sorry--I accidentally hit the POST button while editing this mesage. Disregard this message, and see "Moving files after re-installing Windows WP Pro". ...

Picklists
Say I have a picklist called 'Type', that has 10 values. When I open Account ABC the Type = Customer. Now I delete the value Customer out of the picklist. Then when I open Account ABC, the value is removed out of the picklist BUT all the accounts maked Customer now have a blank/null value displayed as the picklist value. So the text label for Customer is gone. However, if I query via Adv Find and use the integer value that represents Customer, all the accounts still return. Is there any way to delete a value out of the picklist and still keep the text that was deleted as the...

Cant delete messages on server
I was running Outlook 2000 OK. I upgraded to Outlook 2002 that came with pocket PC. Now I cant get my emails messages to delete off the ISP server. On advanced tabs of options the box for LEave a copy of messages on server is unchecked. Could you turn on diagnostic logging (see http://support.microsoft.com/default.aspx?scid=kb;en-us;Q300479) and post the OPMLog.log file (preferably as an attachment) after you've downloaded some mail that wasn't deleted from the server? -- Jeff Stephenson Outlook Development This posting is provided "AS IS" with no warranties, and co...

pst file (again) 2003 Outlook
Why can't I see (or find) my pst file? I am looking in C:\Documents and Settings\Owner\Application Data\Microsoft\Outlook and all I see is Outlook.NK2 and Outlook.srs, no pst file THe program works correctly and when I go to "Data File Management" and double click on my renamed_File, it showe in the path that the file is named Outlook.pst. "TC" <tc@conecap.com> wrote in message news:ef05d458-82b8-41c3-b8a4-c885938dfabe@x21g2000yqa.googlegroups.com... > Why can't I see (or find) my pst file? I am looking in C:\Documents > and Settings\Owner\Applicati...

Script error found during opening Dynamics
I just upgrade to the latest Canada Payroll tax update, when one of the user who sign on to Dynamics after upgrade, she got the "Internet Explorer Script Error" window pop up and saying "An error has occurred in the script on this page" Line: 335 Char: 6 Error: The data is invalid Code: 0 URL: file:///C:/Documents%20and%20Settings/UserA/Local%20Settings/Temp/tmpAD1.tmp I tried to clear all the temp file in this folder but still no luck to get rid of this message during loading Dynamics. I asked the user to click on "Yes" button to continue "D...

Deleting Contacts
I imported thousands of contacts from an excel spreadsheet. It did not work out well, the data did not go into the fields that I thought it would. I am now trying to delete all of these contacts. Is there an easier way to delete other than one at a time? if you change your contact page to display 100 records at a time you can delete a 1000 pretty quickly...how many thousands did you add? -- John O'Donnell Microsoft CRM MVP http://www.microsoft.com/BusinessSolutions/Community/CRMFaqLanding.aspx "Mark" <Mark@discussions.microsoft.com> wrote in message news:73AB942A-28...

VBA script help..Please !!!!
I have been given the script below by Ron De Bruin - and I can't thank him enough, however I'm still scratching my head as it doesn't EXACTLY what I want. I want ALL the built in command bars/toolbars etc that are shown in a normal excel sheet to be hidden when I open my 'Log' - EXCEPT my own personal toolbar named 'LOG' and the 'Worksheet Menu Bar'. 2 problems arise from the script below :- 1) The Worksheet Menu Bar is displayed, but my 'LOG' toolbar is not. 2) If I open another seperate excel sheet, all the same commands have been copied to...