Apple’s variant of ptrace()

Have you ever tried to create a debugger for Mac OS X? It is an adventurous enterprise with lots of unexpected (should I say unforeseeable?) problems. This guy tried and described his adventures in this entertaining post:
His post reminded me of all problems we faced with the first version of the IDA debugger for iMac. They also reminded me of even more convoluted puzzles with the iPhone debugger because ptrace() is broken beyond any hope there (one simple rule: use only PT_TRACEME).
Anyway, if anyone wants to repeat our steps, we are giving away the source code of all debugger modules with the new IDA v5.3: iMac and iPhone debugger codes included. They can certainly help you to avoid some headache and frustration!

3 thoughts on “Apple’s variant of ptrace()”

  1. Great weblog! Check this out, it’s a website talking about information security tools and resources which is being updated daily, you can also subscribe to see the updates on your Google page: (if you want we can exchange links too :) )
    Information Security Software Tools

  2. I was wondering if you had any experience with using the mach calls to implement a simple debugger instead of ptrace. Specifically what I’ve been wondering is on Windows, one can call CreateThread and pass CREATE_SUSPENDED but on MacOSX how would you go about spawning a new process but in the suspended state. I’ve seen some stuff where I could do a fork and then call task_suspend but then how would I load the exe into the new task space? The only other thing that seemed like it might work would be the posix_spawn call. This seems like the first basic thing a debugger would need to do, start a new process and pause it so it can set any breakpoints before running. I was wondering if you had any insight?

  3. It was almost a year ago and my memory may play games on me but AFAIR I could not make it work without ptrace(). The final version of the debugger uses it in 2 locations: to initiate a debugging session after fork() and to resume execution after a debugger event. I tried to replace ptrace() with task_resume() but it had some problems (do not remember what exactly), so I had to switch to ptrace().
    With ptrace(), the problem of creating the debuggee in suspended state does not exist. I do not know/remember if the problem reappears if we work on mach level. I remember checking the kernel code and coming to conclusion that staying on the mach level is impossible. Sorry for not-very-useful response, this is all I remember offhand. Probably things have changed and now it is possible to work on mach level, but is it worth your time?
    Oh yes, maybe it is worth checking the source code of gdbserver for MAC OS X (or iPhone). I do not know how they work, but who knows, maybe they use solely mach calls.

Comments are closed.