c#


LinqToSql InsertOnSubmit memory leak?


I am trying to isolate the source of a "memory leak" in my C# application. This application copies a large number of potentially large files into records in a database using the image column type in SQL Server. I am using a LinqToSql and associated objects for all database access.
The main loop iterates over a list of files and inserts. After removing much boilerplate and error handling, it looks like this:
foreach (Document doc in ImportDocs) {
using (var dc = new DocumentClassesDataContext(connection)) {
byte[] contents = File.ReadAllBytes(doc.FileName);
DocumentSubmission submission = new DocumentSubmission() {
Content = contents,
// other fields
};
dc.DocumentSubmissions.InsertOnSubmit(submission); // (A)
dc.SubmitChanges(); // (B)
}
}
Running this program over the entire input results in an eventual OutOfMemoryException. CLR Profiler reveals that 99% of the heap consists of large byte[] objects corresponding to the sizes of the files.
If I comment both lines A and B, this leak goes away. If I uncomment only line A, the leak comes back. I don't understand how this is possible, as dc is disposed for every iteration of the loop.
Has anyone encountered this before? I suspect directly calling stored procedures or doing inserts will avoid this leak, but I'd like to understand this before trying something else. What is going on?
Update
Including GC.Collect(); after line (B) appears to make no significant change to any case. This does not surprise me much, as CLR Profiler was showing a good number of GC events without explicitly inducing them.
Which operating system are you running this on? Your problem may not be related to Linq2Sql, but to how the operating system manages large memory allocations. For instance, Windows Server 2008 is much better at managing large objects in memory than XP. I have had instances where the code working with large files was leaking on XP but was running fine on Win 2008 server.
HTH
I don't entirely understand why, but making a copy of the iterating variable fixed it. As near as I can tell, LinqToSql was somehow making a copy of the DocumentSubmission inside each Document.
foreach (Document doc in ImportDocs) {
// make copy of doc that lives inside loop scope
Document copydoc = new Document() {
field1 = doc.field1,
field2 = doc.field2,
// complete copy
};
using (var dc = new DocumentClassesDataContext(connection)) {
byte[] contents = File.ReadAllBytes(copydoc.FileName);
DocumentSubmission submission = new DocumentSubmission() {
Content = contents,
// other fields
};
dc.DocumentSubmissions.InsertOnSubmit(submission); // (A)
dc.SubmitChanges(); // (B)
}
}

Related Links

How to get client IP in WCF service in Linux?
SQL CLR aggregate with Hash Group
odata deep-level expand not working
Where Does Generated Connection String Come From?
How to ignore blocking system errors from a child process on windows
getting stuck after sending async request to node.js server
ObjectDisposed Exception on second XPS export
Regex splting by using different seperators
Win32 exception “The operation completed successfully” after setting window owner handle
What's the point of OpenClipboard API?
C# Elements in Dimension
Detect if SQL Server connection is over a VPN
What is the exit code of a C# program that throws an exception out of main?
Always Encrypted column - LINQ query Equals not working
Unity GUIText on Collision c#
If statement on DataTable field doesn't return true

Categories

HOME
macros
teradata
itunesconnect
stanford-nlp
webvr
spss
hiveql
localhost
angularjs-directive
servicenow
normals
olap
ng2-charts
derby
responsive
jsfiddle
virtuemart
md2
combinations
typedef
mattermost
screen-scraping
cloudformation
hierarchical-data
dynamics-ax-2012-r3
delphi-2010
multilanguage
slim-lang
ng2-bootstrap
odp.net
pymunk
stochastic-process
area
linkurious
compiled-query
text-extraction
smooth-streaming
servicebus
line-endings
anonymous-function
yank
openxava
skype-bots
dna-sequence
photoshop-script
jenkins-docker
richtext
r-commander
kendo-scheduler
apache-commons-cli
sqlproj
scikit-image
nsoperationqueue
angular-i18n
std
event-tracking
mojolicious
musicbrainz
pushwoosh
cloud-storage
xml-simple
service-broker
conkeror
nest-initiative
boofcv
adabas
dylan
octokit
auctex
xcode4.6.3
ios6-maps
google-email-audit-api
systrace
discussion-board
recess
overlays
mismatch
pitch-shifting
system-analysis
hp-trim
rpxnow
script-tag

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App