Pitfalls in upgrading a dual boot system

Upgrading a dual-boot system, Windows/Linux, to windows-10 is a tricky business. Just google on "dual boot windows 10 upgrade" and you know what I am talking about.

I was unaware of this and just started the windows-10 installer and had a cup of coffee. When I returned the upgrading process had come to a halt with the terminal only saying 

no such partition
grub rescue
Apparently the windows-10 installer either clobbered the bootloader or the Linux partition or both. I got saved by a program called "Boot-repair" that I ran by booting from a USB stick. Then the windows 10 installation continued and windows-10 is now running fine.

I never managed to recover the Linux partition, but finally just did a fresh Linux installation. As far as running flux under linux everything now seems fine.

A better procedure would have been

  1. backup your Linux as well as windows
  2. uninstall grub, so that the windows bootloader is reinstated
  3. upgrade windows
  4. reinstall Linux + grub

MinGW, OpenMP, and memory management

After installing windows-10, at first Flux refused to work under MinGW, but just reported 
libgomp: Thread creation failed: Resource temporarily unavailable
Later I found I could get things to work by defining 
export OMP_NUM_THREADS=3
This used to be 4, the number of processors. Now it worked again - with 3 parallel threads.

The problem is related with memory management of OpenMP (the multiprocessor handling package) under MinGW. In the previous release, flux7.9.1, scared by reports of program failure due to insufficient stacksize, I had defined 

# extra stack size (0x10000000=256MB) may be needed for OpenMP
XSTACK= -Wl,--stack=0x10000000
This was probably not a wise move. After some searching I found that too large a stacksize defined this way can be harmful. In flux7.9.2 I have reduced the extra stack by a factor of 16 
XSTACK= -Wl,--stack=0x01000000
and now everything works smoothly.

Users are encouraged to experiment themselves with the optimal stacksize. Maybe the parameter defined by XSTACK should be left out altogether and the stacksize manipulated, if necessary, by the environmental variable OMP_STACKSIZE

To permanently modify XSTACK don't edit the makefile itself, but instead the "makedefs.mingw" file used. Then run makemake, followed by makeclean and make.
The lines 

export OMP_NUM_THREADS=.. 
export OMP_STACKSIZE=..
should of course be put in .profile.

By the way in the process I also did a command 

mingw-get upgrade mingw32-libpthreadgc
which installed a newer version of the thread handling don't know if this was essential.

Linux and OpenMP

In Linux I found no problems at all with memory assignment. I can define up to 14000 parallel threads and still everything goes smoothly. Only when I set OMP_NUM_THREADS=16000 things go wrong and I get the Thread creation failed message.

Linux is also much more efficient. Turn-around times for flux jobs are about 70% of the time needed under Windows/MinGW