c#


LinqToSql InsertOnSubmit memory leak?


I am trying to isolate the source of a "memory leak" in my C# application. This application copies a large number of potentially large files into records in a database using the image column type in SQL Server. I am using a LinqToSql and associated objects for all database access.
The main loop iterates over a list of files and inserts. After removing much boilerplate and error handling, it looks like this:
foreach (Document doc in ImportDocs) {
using (var dc = new DocumentClassesDataContext(connection)) {
byte[] contents = File.ReadAllBytes(doc.FileName);
DocumentSubmission submission = new DocumentSubmission() {
Content = contents,
// other fields
};
dc.DocumentSubmissions.InsertOnSubmit(submission); // (A)
dc.SubmitChanges(); // (B)
}
}
Running this program over the entire input results in an eventual OutOfMemoryException. CLR Profiler reveals that 99% of the heap consists of large byte[] objects corresponding to the sizes of the files.
If I comment both lines A and B, this leak goes away. If I uncomment only line A, the leak comes back. I don't understand how this is possible, as dc is disposed for every iteration of the loop.
Has anyone encountered this before? I suspect directly calling stored procedures or doing inserts will avoid this leak, but I'd like to understand this before trying something else. What is going on?
Update
Including GC.Collect(); after line (B) appears to make no significant change to any case. This does not surprise me much, as CLR Profiler was showing a good number of GC events without explicitly inducing them.
Which operating system are you running this on? Your problem may not be related to Linq2Sql, but to how the operating system manages large memory allocations. For instance, Windows Server 2008 is much better at managing large objects in memory than XP. I have had instances where the code working with large files was leaking on XP but was running fine on Win 2008 server.
HTH
I don't entirely understand why, but making a copy of the iterating variable fixed it. As near as I can tell, LinqToSql was somehow making a copy of the DocumentSubmission inside each Document.
foreach (Document doc in ImportDocs) {
// make copy of doc that lives inside loop scope
Document copydoc = new Document() {
field1 = doc.field1,
field2 = doc.field2,
// complete copy
};
using (var dc = new DocumentClassesDataContext(connection)) {
byte[] contents = File.ReadAllBytes(copydoc.FileName);
DocumentSubmission submission = new DocumentSubmission() {
Content = contents,
// other fields
};
dc.DocumentSubmissions.InsertOnSubmit(submission); // (A)
dc.SubmitChanges(); // (B)
}
}

Related Links

How check an email address is valid Lync User?
Install other apps with our setup - vs2010 Desktop App Deployment
Read an Excel (.xlsx) file in Sharepoint C# to a gridview
C# RX (System.Reactive) - Async - Publish an IEnumerable<DataRow> to multiple observing data handers
how to pass value between user control in ASP.NET C#
Expand the %system% directories to full path
37Signals OAuth C# example
Multithreaded Batch Processing (SELECT and UPDATE) [.NET][DB2]
Redirecting new tab OR browser on button click.(Response.Redirect) in asp.net C#
Calling c++ clr dll in c# silverlight
Semaphore WaitOne not operating correctly
Using Ninject 3.0 for a given code
using SqlParameterCollectionExtensions
Populating a Html.Listbox with all ASP.NET users
Something similar to C# .NET Generic List in java
Load arraylist into a TextBox

Categories

HOME
repository
azure-active-directory
alasql
sip
postgresql-9.3
casting
dronekit
mifare
itext7
coordinates
ebay
mediarecorder
ms-access-2010
aggregate-functions
minimax
velocity
saucelabs
template10
summernote
webgl-earth
typedef
bootstrap-accordion
continuous-fourier
stack-trace
mex
marionette
typeorm
delphi-2010
exacttarget
keil
snapkit
node-soap
microstrategy
n1ql
perf
mapquest
iterm2
jslider
aspose.pdf
ui-grid
sknode
socketcluster
file-descriptor
flume-ng
qt-quick
qtwebkit
spring-ide
togglebutton
cloudconvert
cancan
audio-fingerprinting
htop
codesniffer
tsc
iban
acceleo
suitesparse
msiexec
easy-thumbnails
boost-range
syslog4j
recurring
transaction-isolation
angular-i18n
razorengine
camellia
cascading
overlapping
gulp-cdnizer
serial-communication
joomla3.3
borland-c++
euro
photogrammetry
genome
genero
mri
grunt-contrib-connect
diffmerge
indic
sat4j
xpand
dateadd
mixradio
facebook-java-api
kuka-krl
vbaccelerator
code-complete
xcode5.1
friendly-id
source-depot
http-compression
select2-rails
magicsuggest
pymel
microsoft-speech-platform
correctness
m3u
optimistic-concurrency
win-prolog
unordered-set
joomla1.7
umfpack
clgeocoder
openwrap
jexcelapi
functional-specifications
cryptolicensing
hs-err
noise-reduction
marathontesting
netbeans7.0
versions
morfik
ffdshow
vista64

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App