IDA 7.2: Qt 5.6.3 configure options & patch

A handful of our users have already requested information regarding the Qt 5.6.3 build, that is shipped with IDA 7.2.

Configure options

Here are the options that were used to build the libraries on:

  • Windows: ...\5.6.3\configure.bat "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "win32-msvc2015" "-opengl" "desktop" "-prefix" "C:/Qt/5.6.3-x64"
    • Note that you will have to build with Visual Studio 2015 or newer, to obtain compatible libs
  • Linux: .../5.6.3/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "linux-g++-64" "-developer-build" "-fontconfig" "-qt-freetype" "-qt-libpng" "-glib" "-qt-xcb" "-dbus" "-qt-sql-sqlite" "-gtkstyle" "-prefix" "/usr/local/Qt/5.6.3-x64"
  • Mac OSX: .../5.6.3/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "macx-clang" "-debug-and-release" "-fontconfig" "-qt-freetype" "-qt-libpng" "-qt-sql-sqlite" "-prefix" "/Users/Shared/Qt/5.6.3-x64"

patch

In addition to the specific configure options, the Qt build that ships with IDA includes the following patch. You should therefore apply it to your own Qt 5.6.3 sources before compiling, in order to obtain similar binaries (patch -p 1 < path/to/qt-5_6_3_full-IDA72.patch)

Note that this patch should work without any modification, against the 5.6.3 release as found there. You may have to fiddle with it, if your Qt 5.6.3 sources come from somewhere else.

IDAPython: wrappers are only wrappers

Intended audience

IDAPython developers who enjoy the occasional headache, leaky abstraction enthousiasts, or simply the curious.

TL;DR

IDAPython wraps C++ types, and the lifecycle of C++ objects (and in particular members of larger objects) is not necessarily the same as that of the Python wrapper object that is wrapping it.

The problem

One of our users reported IDA crashes when an IDAPython script of theirs. The user came up with a very simple way to reproduce the issue (thank you!), showing that this had to do with accessing the parents member of a ida_hexrays.ctree_visitor_t instance.

Here is (an even more simplified version of) the script the user sent us:

from ida_hexrays import *

my_parents = None

class my_visitor_t(ctree_visitor_t):
    def __init__(self, func):
        ctree_visitor_t.__init__(self, CV_PARENTS)

    def visit_expr(self, i):
        global my_parents
        if self.parents is not None:
            my_parents = self.parents
        return 0


def my_cb(event, *args):
    if event == hxe_print_func:
        f = args[0]
        my_visitor_t(f).apply_to(f.body, None)
        import gc
        gc.collect()
        my_parents.front() # will crash
    return 0

install_hexrays_callback(my_cb)

Note: I threw a gc.collect() in there, to make crashes more likely.

The script above is provided in its entirety for the sake of completeness, but really the important lines are only the following:

    def visit_expr(self, i):
        global my_parents
        if self.parents is not None:
            my_parents = self.parents

    (...)

        my_visitor_t(f).apply_to(f.body, None)
        my_parents.front() # will crash

Details, details, details

Since this issue is non-trivial, I’ll try and provide a step-by-step explanation, hopefully as clear as can be, by annotating the important lines of code mentioned above:

        my_visitor_t(f)

Create a my_visitor_t instance. That is a subclass of the ctree_visitor_t type, which means it eventually extends a C++ object of type ctree_visitor_t.

When the underlying C++ ctree_visitor_t object is created, its member named parents (a ctree_items_t vector) is initialized. For the sake of the example, let’s say the C++ ctree_visitor_t instance is located at memory 0x1000 and the parents member is placed at memory 0x100C.

                       .apply_to(f.body, None)

Call ctree_visitor_t::apply_to. Thanks to SWiG “magic”, C++ virtual method calls will be properly redirected and our my_visitor_t.visit_expr method will be called for each cexpr_t in the tree, as expected.

        if self.parents is not None:

Access self.parents. This will create a Python wrapper object. The key here is to understand that it’s a wrapper object which is backed by the real, C++ ctree_items_t instance.

For example, any access to the object returned by self.parents, will in fact translate to an access into the C++ ctree_items_t vector, so if one were to write, e.g., self.parents.size() (or even len(self.parents)), it’s actually the real underlying C++ ctree_items_t instance’s size() method that will end up being called.

            my_parents = self.parents

Another access to self.parents, and another Python wrapper will be created (once again backed by the actual ctree_items_t vector)

[Note: the fact that another wrapper is created is not a problem (in fact since it went out of scope, the previous wrapper might already have been garbage collected!)]

Once again, for the sake of the example, let’s say the wrapping PyObject instance is placed in memory, at 0xB000.
That wrapper is then bound to the global variable my_parents, causing its python refcount to increase to 2. Past that line, the refcount will drop back to 1 (again, because of scope logic), which means that Python wrapper object will remain alive.

[...apply_to() returns, and we are now back to the `my_cb` function...]

At this point, it’s likely my_visitor_t(f) has just been garbage collected since nobody keeps a reference to it.

That means:

  • the my_visitor_t instance has been destroyed, which means
  • the underlying ctree_visitor_t C++ object located at memory 0x1000 has been deleted, which in turn means
  • its parents object, which was located at memory 0x100C, is now invalid

                my_parents.front()
    

We are now calling front() on the my_parents Python object. If you recall, that my_parents object is a Python wrapper object located in memory at 0xB000. That wrapper object still has a refcount of (at least) 1, and is thus alive.

What is not quite alive anymore, however, is the actual C++ ctree_items_t vector, which was deleted as part of deleting the C++ ctree_visitor_t it belonged to.

In other words, we have a perfectly valid Python wrapper object, that has a dangling pointer to a member of a freshly-deleted C++ object.

The solution

The solution is, in terms of effort, rather simple: make a copy of the vector:

-            my_parents = self.parents
+            my_parents = ctree_items_t(self.parents)

since it doesn’t belong to the C++ ctree_visitor_t object, this copy won’t be thrashed when it is deleted.

IDA on non-OS X/Retina Hi-DPI displays

The problem

Some users running IDA on Windows & Linux X11 platforms with Hi-DPI displays, have reported that IDA looks rather odd: the navigator bar is too narrow, the text under it gets truncated, and there is overall feeling of packing & clumsiness:

  • Windows:
  • Linux X11:

Looking closely, one can notice the following issues (probably not an exhaustive list)

Note: this should not happen and shouldn’t apply for OS X users running IDA on Retina displays. Nor should it happen (but we didn’t get a chance to test this) on non-X11 Linux display managers, such as Wayland.

Fix / mitigation:

On Linux X11 & Windows, if you are using Hi-DPI monitors and IDA looks somewhat like it does in the above screenshots, please try setting the environment variable QT_AUTO_SCREEN_SCALE_FACTOR to 1:

E.g., on Linux/X11:

~# export QT_AUTO_SCREEN_SCALE_FACTOR=1
~# path/to/ida my.idb

IDA should now look more pleasant:

  • Windows:
  • Linux X11:

Some things are still not perfect (e.g., checkboxes might remain small), but IDA definitely looks better.

Please give it a try!

Gory details

When one applies scaling/zooming, either in Windows and Linux X11, that will have the effect of causing the OS to return scaled values for font metrics when queried using point sizes (which is almost always the case.)

For example, when the font metrics for a font of size 12pt are requested by a Qt application, instead of returning 14 pixels as it would on a non-scaled system, the operating system will instead return 28 pixels on a 200% scaled one (in other words, this is essentially a font database-related feature).

That will, in turn, have the net effect of causing Qt to compute layout of the surrounding widgets according to those scaled font metrics, which explains why the text is (for the most part) not truncated.

However, what applying scaling does not do, is tell Qt that it should scale all other pixel measurements by that scale factor.

Consequently, paddings, margins, scrollbars, etc… all have uncomfortably small dimensions, especially when compared to text.

The QT_AUTO_SCREEN_SCALE_FACTOR environment variable is an opt-in that program users can define, in order to control how the program should look. It will in essence instruct Qt to perform automatic scaling of (non-font-related) graphical operations according to the pixel density of the screen(s).

More information can be found on Qt’s website.

Why is this not needed under OS X + Retina?

This is not needed under OS X + Retina, because Qt does not need to perform any kind of scaling there: the scaling is performed by the drawing primitives of the OS itself, and is entirely transparent to the application.

(In fact, an OSX application doesn’t even work with the real screen geometry, but rather with an “abstract” coordinate system, normalizing pixel sizes across screen densities.)

IDA 7.1: Qt 5.6.3 configure options & patch

A handful of our users have already requested information regarding the Qt 5.6.3 build, that is shipped with IDA 7.1.

Configure options

Here are the options that were used to build the libraries on:

  • Windows: ...\5.6.3\configure.bat "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "win32-msvc2015" "-opengl" "desktop" "-prefix" "C:/Qt/5.6.3-x64"
    • Note that you will have to build with Visual Studio 2015, to obtain compatible libs
  • Linux: .../5.6.3/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "linux-g++-64" "-developer-build" "-fontconfig" "-qt-freetype" "-qt-libpng" "-glib" "-qt-xcb" "-dbus" "-qt-sql-sqlite" "-gtkstyle" "-prefix" "/usr/local/Qt/5.6.3-x64"
  • Mac OSX: .../5.6.3/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "macx-g++" "-debug-and-release" "-fontconfig" "-qt-freetype" "-qt-libpng" "-qt-sql-sqlite" "-prefix" "/Users/Shared/Qt/5.6.3-x64"

patch

In addition to the specific configure options, the Qt build that ships with IDA includes the following patch. You should therefore apply it to your own Qt 5.6.3 sources before compiling, in order to obtain similar binaries (patch -p 1 < path/to/qt-5_6_3_full-IDA71.patch)

Note that this patch should work without any modification, against the 5.6.3 release as found there. You may have to fiddle with it, if your Qt 5.6.3 sources come from somewhere else.

IDAPython: namespacing for plugins, loaders & processor modules.

Intended audience

IDAPython plugins, loaders or processor modules developers.

The problem

Until now, IDAPython would load all loaders, processor modules & plugins in the '__main__' module.

This causes namespace pollution, which can sometimes leads to very obscure errors.

The solution

Starting with version 7.1, IDA will import plugins, loaders & processor modules in their own, separate Python modules.

The names of those Python modules is derived from the plugin, loader or processor module’s file name.

E.g.,

  • a plugin named myplg.py will now be imported into its own '__plugins__myplg' Python module.
  • a loader named myldr.py will now be imported into its own '__loaders__myldr' Python module.
  • a processor module named myprc.py will now be imported into its own '__procs__myprc' Python module.

My plugin/loader/processor module complains about unexisting variables/functions!

It very likely means that the code in question was, in effect, relying on some other code being loaded before it, and that was “polluting” the '__main__' module in a way that was very fortunate from its point-of-view (a happy coincidence, if you will.)

The solution is of course to fix the code in question so that it imports everything it needs, before making use of it.

However…

I’m in a hurry

As a temporary workaround, you can set NAMESPACE_AWARE to NO in cfg/python.cfg.

This will cause IDA to revert to the old behavior, where the '__main__' module is shared across all plugins, loaders, processor modules & user scripts.

However, please note that support for NAMESPACE_AWARE will be removed sometimes in the future.

IDA 7.0: Qt 5.6.0 configure options & patch

A handful of our users have already requested information regarding the Qt 5.6.0 build, that is shipped with IDA 7.0.

Configure options

Here are the options that were used to build the libraries on:

  • Windows: ...\5.6.0\configure.bat "-nomake" "tests" "-qtnamespace" "QT"
    "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "win32-msvc2015" "-opengl" "desktop" "-prefix" "C:/Qt/5.6.0-x64"

    • Note that you will have to build with Visual Studio 2015, to obtain compatible libs
  • Linux: .../5.6.0/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "linux-g++-64" "-developer-build" "-fontconfig" "-qt-freetype" "-qt-libpng" "-glib" "-qt-xcb" "-dbus" "-qt-sql-sqlite" "-gtkstyle" "-prefix" "/usr/local/Qt/5.6.0-x64"
  • Mac OSX: .../5.6.0/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "macx-g++" "-debug-and-release" "-fontconfig" "-qt-freetype" "-qt-libpng" "-qt-sql-sqlite" "-prefix" "/Users/Shared/Qt/5.6.0-x64"

patch

In addition to the specific configure options, the Qt build that ships with IDA includes the following patch. You should therefore apply it to your own Qt 5.6.0 sources before compiling, in order to obtain similar binaries.

Note that this patch should work without any modification, against the 5.6.0 release as found there. You may have to fiddle with it, if your Qt 5.6.0 sources come from somewhere else.

IDA 6.95: Qt 5.6.0 configure options & patch

A handful of our users have already requested information regarding the Qt 5.6.0 build, that is shipped with IDA 6.95.

Configure options

Here are the options that were used to build the libraries on:

  • Windows: ...\5.6.0\configure.bat "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "win32-msvc2015" "-opengl" "desktop" "-prefix" "C:/Qt/5.6.0"
    • Note that you will have to build with Visual Studio 2015, to obtain compatible libs
  • Linux: .../5.6.0/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "linux-g++-32" "-developer-build" "-fontconfig" "-qt-freetype" "-qt-libpng" "-glib" "-qt-xcb" "-dbus" "-qt-sql-sqlite" "-gtkstyle" "-prefix" "/usr/local/Qt/5.6.0"
  • Mac OSX: .../5.6.0/build/../configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "macx-g++-32" "-debug-and-release" "-fontconfig" "-qt-freetype" "-qt-libpng" "-qt-sql-sqlite" "-prefix" "/Users/Shared/Qt/5.6.0"

patch

In addition to the specific configure options, the Qt build that ships with IDA includes the following patch. You should therefore apply it to your own Qt 5.6.0 sources before compiling, in order to obtain similar binaries.

Note that this patch should work without any modification, against the 5.6.0 release as found there. You may have to fiddle with it, if your Qt 5.6.0 sources come from somewhere else.

Installing IDA 6.95 on Linux

IDA is still, as of this writing (August 9th, 2015), a 32-bit application and both IDA & its installer(*) require certain 32-bit libraries to be present on your Linux system before they can run.

Here is the list of commands you will have to run in order to install those dependencies, for the following systems:

  • Debian & derivative systems such as Ubuntu, Xubuntu, …
  • Red Hat Enterprise Linux 7.2 (and likely other versions as well)
Note: we cannot possibly install & try IDA on all flavors/versions of all Linux distributions, but we will do our best to update this post with relevant information, whenever we learn of a distribution requiring special attention.

(*) that is: if you want the installer to run a graphical interface, instead of a command-line one.

Debian & Ubuntu

Common dependencies

The following should allow IDA to run on most Linux systems deriving from Debian distributions:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6-i686:i386 libexpat1:i386 libffi6:i386 libfontconfig1:i386 libfreetype6:i386 libgcc1:i386 libglib2.0-0:i386 libice6:i386 libpcre3:i386 libpng12-0:i386 libsm6:i386 libstdc++6:i386 libuuid1:i386 libx11-6:i386 libxau6:i386 libxcb1:i386 libxdmcp6:i386 libxext6:i386 libxrender1:i386 zlib1g:i386 libx11-xcb1:i386 libdbus-1-3:i386 libxi6:i386 libsm6:i386 libcurl3:i386 

# In addition, it is also necessary to install the
# following packages, for IDA to present a usable
# & well-integrated GUI on many Debian & Ubuntu
# desktops. If the set of dependencies above are
# not enough to obtain a slick UI, please 
# install the following:

sudo apt-get install libgtk2.0-0:i386 gtk2-engines-murrine:i386 gtk2-engines-pixbuf:i386 libpango1.0-0:i386
Note: one of our users kindly warned us that the package “libpng12-0:i386” has been replaced by “libpng16-16:i386” in Debian Stretch repositories. Consequently, you may have to change the before-last command above to:

sudo apt-get install libc6-i686:i386 libexpat1:i386 libffi6:i386 libfontconfig1:i386 libfreetype6:i386 libgcc1:i386 libglib2.0-0:i386 libice6:i386 libpcre3:i386 libpng16-16:i386 libsm6:i386 libstdc++6:i386 libuuid1:i386 libx11-6:i386 libxau6:i386 libxcb1:i386 libxdmcp6:i386 libxext6:i386 libxrender1:i386 zlib1g:i386 libx11-xcb1:i386 libdbus-1-3:i386 libxi6:i386 libsm6:i386 libcurl3:i386 

Red Hat Enterprise Linux 7.2

IDA will require the following packages to be installed, in order to run properly on RHEL 7.2 (and probably any other RPM-based distribution) :

redhat-lsb-core.i686
glib2.i686
libXext.i686
libXi.i686
libSM.i686
libICE.i686
freetype.i686
fontconfig.i686
dbus-libs.i686 
libgnomeui:i686

Arch Linux

While the author of this post is not quite familiar with Arch Linux, one of our users reported that adding the required dependencies can be performed through the AUR package that can be found at: https://aur.archlinux.org/packages/ida-pro-6.4/.

(And then, installing a 32-bit system-wide interpreter (i.e., if one favors that option over the default that consists of using the Python runtime shipped with IDA), should be performed by installing the package ‘lib32-python2’.)

IDA 6.9, Mac OS X, ‘random’ crashes

Intended audience

IDA 6.9 users on Mac OS X, who have suffered seemingly-apparent crashes while using IDA.

The problem

The Qt 5.4.1 libraries shipped with IDA 6.9 suffer from the following bug: https://bugreports.qt.io/browse/QTBUG-44708, which was apparently fixed in Qt 5.5.0.

If, when IDA crashes, you ever spotted a backtrace that looks like the following:

frame #0: 0x00000000
frame #1: 0x00d8a50d QtGui'QT::QTextEngine::shapeText(int) const + 1187
frame #2: 0x00d8b517 QtGui'QT::QTextEngine::shape(int) const + 1199
frame #3: 0x00d8c977 QtGui'QT::QTextEngine::width(int, int) const + 155
frame #4: 0x00d73571 QtGui'QT::QFontMetricsF::width(QT::QString const&) const + 163
frame #5: 0x00041184 idaq'___lldb_unnamed_function853$$idaq + 420
...

then you’ve been a victim of this rather tiresome issue.

(note: frame #0 doesn’t quite matter; the 2nd line, QT::QTextEngine::shapeText(int), is the important one)

The solution

We have applied the patch mentionned in the Qt bugreport & re-built the libqcocoa.dylib Qt platform support.

You will have to:

  1. download & unzip the following archive: qcocoa.zip.
  2. verify the shasum of the libqcocoa.dylib file:
    $ shasum libqcocoa.dylib 
    afcf3603f593776c6f39f41f81e98843897cf0ed  libqcocoa.dylib
    
  3. place the libqcocoa.dylib binary instead of the one in /path/to/IDA_6.9/idaq.app/Contents/Plugins/

Once that is done, those crashes shouldn’t happen anymore.

A big, big thank you to Willem Jan Hengeveld & Vladimir Putin, who have reported this!

Beware: IDA C++ plugins, Qt 5.x, QStringLiteral: crash at exit-time

Intended audience

IDA C++ plugin authors, who wish to link such plugins against Qt 5.x libraries.

The problem

One of our customers, Aliaksandr Trafimchuk, recently reported that whenever IDA was run with a plugin of his that links against the Qt libraries that we ship, IDA would crash at exit-time (at least on Windows.)

Aliaksandr already did most of the work of figuring out exactly what was causing the crash, and even had a work-around (more like a kludge, as he pointed out, really) for it, but he still wanted to let us know about it so we are aware of the problem & perhaps can communicate about it.

The crash is an access violation, in an area of memory that doesn’t seem to be mapped by any stack, heap, DLL code or data.

The stack reveals that the crash happens at QCoreApplication::~QCoreApplication()-time (i.e., at application exit), when the QFontCache is freeing/releasing its entries:

  Qt5Core.dll!QT::QSettingsGroup::~QSettingsGroup()
  Qt5Gui.dll!QT::QMapNode::destroySubTree()
  Qt5Gui.dll!QT::QFontCache::clear()
  Qt5Gui.dll!QT::QFontCache::~QFontCache()
  [External Code] 
  Qt5Gui.dll!QT::QThreadStorage::deleteData(void * x)
  Qt5Core.dll!QT::QThreadStorageData::set(void * p)
  Qt5Gui.dll!QT::QFont::cleanup()
  Qt5Gui.dll!QT::QGuiApplicationPrivate::~QGuiApplicationPrivate()
  [External Code] 
  Qt5Core.dll!QT::QObject::~QObject()
  Qt5Core.dll!QT::QCoreApplication::~QCoreApplication()
  idaq.exe!013426c5() Unknown
  (...)

Why does that happen?

Our customer’s plugin uses a UI description file, that needs to be processed by Qt’s uic (UI-compiler). The generated code contains lines such as these:

label = new QLabel(TestDialog);
label->setObjectName(QStringLiteral("label"));
QFont font;
font.setFamily(QStringLiteral("Comic Sans MS"));

Note the use of QStringLiteral.

The QStringLiteral type

This is an optimization that came in Qt 5.x, and that causes actual QString instances to be laid out in the .rodata section of the program (together with a special refcount value that is -1, meaning “don’t touch this refcount”.)

Although at exit-time, this “static const” in-.rodata-QString instance wouldn’t be modified (because of the -1 refcount), simply reading it will cause a crash, since the section holding it has been removed from memory.

This is a known limitation/problem, too: https://bugreports.qt.io/browse/QTBUG-46880

The plugin lifecycle

This is where the problem lies: at exit-time, IDA will:

  1. unload plugins
  2. proceed until its QCoreApplication goes out of scope, which will perform (among other things) the QFontCache cleanup.
  3. alas, at that time, the QFontCache still refers to literal QString data, in a section that is now gone (it was discarded at #1)

In fact, Qt expects that any binary that uses Qt libraries should remain in memory, so that some optimizations (such as the QStringLiteral) will continue to work. That’s why, when Qt unloads some of its own plugins, it doesn’t really unload those from memory.

Although the Qt library maintainers consider that having such limitations on binaries that link against Qt is acceptable, I personally hope they try to keep those restrictions as minimal as possible.

The solution

In any case, concerning this QStringLiteral issue, we have a way out: at compilation-time, pass the compiler the following flag: -DQT_NO_UNICODE_LITERAL

This will turn the QStringLiteral() expression into a QString::fromUtf8(), which will allocate the memory on the heap and the plugin should work just fine.


Another possible solution

Another possibility reported by an IDA user (but untested by us), is to add the following after the Qt headers #include directives:

#undef QStringLiteral
#define QStringLiteral(_str) _str

With this method, the literal C-style string will be implicitly converted to a QString, using the default conversion rules.


Footnotes

The kludge (which is Windows-specific) consists of calling LoadLibrary(szMyPluginFilePath), thereby somewhat artificially incrementing the refcount of his plugin, which will cause it to remain in memory & thus the ~QFontCache cleanup will succeed.