So I am creating a customer indexing program for my company and I have basically everything coded and working except I want the indexing program to watch the user-specified indexed directories and update the underlying data store on the fly to help eliminate the need for full indexing often.

I coded everything in WPF/C# with an underlying SQLite database and I am sure the folder watchers would work well under "non-heavy loads", but the problem is we use TortoiseSVN and when the user does an SVN Update, etc. that creates a heavy file load the FileSystemWatcher and SQLite updates just can't keep up (even with the max buffer size). Basically I am doing a database insert every time the watcher event is hit.

So my main question is...does anyone have suggestions on how to implement this file watcher to handle such heavy loads?

Some thoughts I had were: (1) Create a staging collection for all the queries and use a timer and thread to insert the data later on (2) Write the queries to a file and use a timer thread later on for the insert

Help....

You want to buffer the data received from your file watch events in memory. Thus when receiving the events from your registered file watchers, you can accumulate them in memory as fast as possible for a burst of activity. Then on a separate process or thread, you read them from your in memory buffer and do whatever you need for persistent storage or whatever process is more time intensive.