ArchiveOrangemail archive

Technical discussion on Darwin user-level software. This does not include Carbon, Cocoa, or Darwin Streaming Server.


darwin-userlevel.lists.apple.com
(List home) (Recent threads) (100 other Apple lists)

Subscription Options

  • RSS or Atom: Read-only subscription using a browser or aggregator. This is the recommended way if you don't need to send messages to the list. You can learn more about feed syndication and clients here.
  • Conventional: All messages are delivered to your mail address, and you can reply. To subscribe, send an email to the list's subscribe address with "subscribe" in the subject line, or visit the list's homepage here.
  • Low traffic list: less than 3 messages per day
  • This list contains about 152 messages, beginning Oct 2009
  • 0 messages added yesterday
Report the Spam
This button sends a spam report to the moderator. Please use it sparingly. For other removal requests, read this.
Are you sure? yes no

Possible to get Unified Buffer Cache info on iOS?

Ad
David Hoerl 1332690225Sun, 25 Mar 2012 15:43:45 +0000 (UTC)
I have an app that makes heavy use of mmap for large image files. Its 
crashing (or getting killed) on the 256M devices with no crash report or 
other messages.

My mmap calls all set MAP_NOCACHE.

I've found that by calling fcntl(F_FULLSYNC) on each file after I've 
unmapped the memory stops the crashes. I'd like to delve more into 
what's going on, but have not found anything amiss in my app's memory 
footprint using task_info or host_statistics().

All I can of is that I've just overloaded the UBC. My app gets killed or 
dies AFTER it munmapped's its memory. If I flush at that point, then no 
crash.

Suggestions or pointers most welcome!

David
Nehemiah I. Dacres 1332726723Mon, 26 Mar 2012 01:52:03 +0000 (UTC)
you can be sure its not being killed by intercepting SIGTERM to print a debug message before exiting.
David Hoerl 1332769279Mon, 26 Mar 2012 13:41:19 +0000 (UTC)
On 3/25/12 9:51 PM, Nehemiah I. Dacres wrote:
>   you can be sure its not being killed by intercepting SIGTERM to print
> a debug message before exiting.Well, I tried that no msg. I also tried setting a signal handler for 
every signal from 1 to 31 - none fired.

Then, I turned of fcntl(F_FULL_SYNC), and replaced it with 
msync(MS_SYNC) just before every munmap, and the crash stopped 
(execution time stayed exactly where it was using F_FULLFSYNC).

Finally, I changed MS_SYNC to MS_ASYNC, and the app still worked with an 
execution time exactly as in the MS_SYNC case ?!?!?!

I'm testing every mmap/msync/munmap return code and asserting if not 0. 
I logged every mmap and munmap to assure myself that in fact I was 
properly unmapping the proper address/size pair. I even tried in a bogus 
assert() to assure myself the asserts were working.

What I glean from all this is as follows:

- a program can write to files much faster than the file system can sync 
it to disk (no surprise)

- the OS must assign the memory consumed in the Unified Buffer Cache to 
the process consuming it (i.e. writing the files)

- when the combined memory consumption (ubc and heap) exceed a certain 
size, the process gets clobbered - or perhaps if the ubc consumption 
alone exceeds a threshold the process gets terminated. [I log UIKit 
memory messages and have seen them in this same code under different 
circumstances.]

- probably the OS code that deals with a memory hog using large numbers 
of ubc buffers is different from the code that detects heap based memory 
hogs, and the way the process gets clobbered is just more opaque. This 
is why I get no signal, there is no log message, lldb/gdb cannot detect 
the termination, etc.

If I had the means to probe the ubc as to my consumption, even if was 
crude, I'd have the means to throttle my usage to some "safe" value. As 
it stands, with this being apparently so opaque, my only option is to 
sync all the time (for a much poorer user experience).


Thus, any ideas on how I might uncover my actual UBC usage most welcome 
(or uncovering how much is too much :-) )
Nehemiah I. Dacres 1332775212Mon, 26 Mar 2012 15:20:12 +0000 (UTC)
you can probe the OS code with Dtrace. make sure you get rid of some of that debugging code before you ship\ 

apparently, and I qoute "The Unified Buffer Cache implementation in Mac OS X differs from that found in FreeBSD." but it doesn't say how
https://developer.apple.com/library/mac/#docu...
Home | About | Privacy