[eigen] Help to compile/run unit tests for next 3.3.5 release

[Gael Guennebaud]

Hi everybody,

it would be extremely helpful if some of you could build and run our unit
tests on the 3.3 branch with MSVC compilers. Basically all you need to do
is:

hg clone https://bitbucket.org/eigen/eigen eigen3.3
cd eigen3.3
hg up 3.3
mkdir build
cd build
cmake ..
ctest -C Release -D Experimental

The outcome will be visible to us there:
http://manao.inria.fr/CDash/index.php?project=Eigen3.3

There more details there: http://eigen.tuxfamily.org/index.php?title=Tests

thanks for your help,
gael


PS: For the long story, I used to have continuous builds with MSVC but all
my VMs have been deleted by an error of our sysadmin, and I struggle to
recreate them.

[Gael Guennebaud]

Post by Gael Guennebaud
[...]
ctest -C Release -D Experimental

there is small issue, the last line should be:

ctest -D Experimental

For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.

gael

[Patrik Huber]

Hi Gael,

I've started builds on VS2017 15.5.1 and VS2017 15.6.0 Preview 1.0 with and
without -DEIGEN_TEST_CXX11=ON (I thought that might be useful) between 12
and 2pm, one with -j 9 and the other one without. Both have been running
since and haven't completed yet.
One is currently at "Size: 1299K", the other one at "2050K". I read on the
wiki that a full rebuild can take up to three hours, but particularly with
-j 9 and a good machine (SSD, recent quad-core i7), I am surprised. Is this
to be expected?

I've essentially followed your instructions and then

cmake .. -G "Visual Studio 15 2017 Win64"
ctest -C Release -D Experimental -j 9

Post by Gael Guennebaud
ctest -D Experimental

So that means no "-C Release"? What will that run?


I'm happy to let them run & finish or restart them with modified commands,
just let me know.

Kind regards,

Patrik


--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom

Web: www.patrikhuber.ch
Mobile: [phone]

On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud <

Post by Gael Guennebaud
[...]
ctest -C Release -D Experimental

ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.
gael

[Gael Guennebaud]

Post by Patrik Huber
Hi Gael,
I've started builds on VS2017 15.5.1 and VS2017 15.6.0 Preview 1.0 with
and without -DEIGEN_TEST_CXX11=ON (I thought that might be useful) between
12 and 2pm, one with -j 9 and the other one without. Both have been running
since and haven't completed yet.
One is currently at "Size: 1299K", the other one at "2050K". I read on the
wiki that a full rebuild can take up to three hours, but particularly with
-j 9 and a good machine (SSD, recent quad-core i7), I am surprised. Is this
to be expected?

Thanks a lot for starting the builds. I run it with MSVC 2012 in a VM
without multithreading and it completed in 2 to 3 hours max. The
build/test/Debug folder is about 6GB. So:
1 - make sure you've updated to 3.3 (the devel branch is more involved,
because there are more tests!)
2 - To speed up builds you can add /DEIGEN_STRONG_INLINE=inline in
CMAKE_CXX_FLAGS
3 - "ctest -j 9" won't speedup the builds, but only the run of the tests.

So for faster parallel builds (4 processus):

cmake . -DCMAKE_CXX_FLAGS="/MP4 /DEIGEN_STRONG_INLINE=inline"

Post by Patrik Huber
I've essentially followed your instructions and then
cmake .. -G "Visual Studio 15 2017 Win64"
ctest -C Release -D Experimental -j 9

Post by Gael Guennebaud
ctest -D Experimental

So that means no "-C Release"? What will that run?

all tests will be run instead of the ones in eigen/test only (in theory,
here none were run, which is why I recommended to remove this option).

In case of problems, you can rerun the tests without rebuilding binaries
that are already up-to-date with:

ctest -D Experimental --build-noclean -VV

-VV is useful to see what's going on...

thanks again.

Post by Patrik Huber
I'm happy to let them run & finish or restart them with modified commands,
just let me know.
Kind regards,
Patrik
--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom
Web: www.patrikhuber.ch
Mobile: [phone] <+44%207482%20633934>

On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud <

Post by Gael Guennebaud
[...]
ctest -C Release -D Experimental

ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.
gael

--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom
Web: www.patrikhuber.ch
Mobile: [phone] <+44%207482%20633934>

[Ian Bell]

Any interest in VC2008? I can probably run those on my mac mini. VC2008
is still used for instance in Python 2.7

Post by Gael Guennebaud

Post by Patrik Huber
Hi Gael,
I've started builds on VS2017 15.5.1 and VS2017 15.6.0 Preview 1.0 with
and without -DEIGEN_TEST_CXX11=ON (I thought that might be useful) between
12 and 2pm, one with -j 9 and the other one without. Both have been running
since and haven't completed yet.
One is currently at "Size: 1299K", the other one at "2050K". I read on
the wiki that a full rebuild can take up to three hours, but particularly
with -j 9 and a good machine (SSD, recent quad-core i7), I am surprised. Is
this to be expected?

Thanks a lot for starting the builds. I run it with MSVC 2012 in a VM
without multithreading and it completed in 2 to 3 hours max. The
1 - make sure you've updated to 3.3 (the devel branch is more involved,
because there are more tests!)
2 - To speed up builds you can add /DEIGEN_STRONG_INLINE=inline in
CMAKE_CXX_FLAGS
3 - "ctest -j 9" won't speedup the builds, but only the run of the tests.
cmake . -DCMAKE_CXX_FLAGS="/MP4 /DEIGEN_STRONG_INLINE=inline"

Post by Patrik Huber
I've essentially followed your instructions and then
cmake .. -G "Visual Studio 15 2017 Win64"
ctest -C Release -D Experimental -j 9

Post by Gael Guennebaud
ctest -D Experimental

So that means no "-C Release"? What will that run?

all tests will be run instead of the ones in eigen/test only (in theory,
here none were run, which is why I recommended to remove this option).
In case of problems, you can rerun the tests without rebuilding binaries
ctest -D Experimental --build-noclean -VV
-VV is useful to see what's going on...
thanks again.

Post by Patrik Huber
I'm happy to let them run & finish or restart them with modified
commands, just let me know.
Kind regards,
Patrik
--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom
Web: www.patrikhuber.ch
Mobile: [phone] <+44%207482%20633934>

On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud <

Post by Gael Guennebaud
[...]
ctest -C Release -D Experimental

ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.
gael

--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom
Web: www.patrikhuber.ch
Mobile: [phone] <+44%207482%20633934>

[Gael Guennebaud]

Post by Ian Bell
Any interest in VC2008? I can probably run those on my mac mini. VC2008
is still used for instance in Python 2.7

We do not support 2008 anymore and I expect lots of compilation errors, but
who knows...

[Patrik Huber]

Looks like it worked now, excellent! It took 2-3 hours or something like
that now and it is uploaded.

I'll run 15.6.0 Preview 1.0 now too.
Let me know if you're interested in running it with any other
settings/flags, as long as it's VS2017 or 2017 Preview I'm happy to run.

Best wishes,

Patrik

Post by Gael Guennebaud

Post by Patrik Huber
Hi Gael,
I've started builds on VS2017 15.5.1 and VS2017 15.6.0 Preview 1.0 with
and without -DEIGEN_TEST_CXX11=ON (I thought that might be useful) between
12 and 2pm, one with -j 9 and the other one without. Both have been running
since and haven't completed yet.
One is currently at "Size: 1299K", the other one at "2050K". I read on
the wiki that a full rebuild can take up to three hours, but particularly
with -j 9 and a good machine (SSD, recent quad-core i7), I am surprised. Is
this to be expected?

Thanks a lot for starting the builds. I run it with MSVC 2012 in a VM
without multithreading and it completed in 2 to 3 hours max. The
1 - make sure you've updated to 3.3 (the devel branch is more involved,
because there are more tests!)
2 - To speed up builds you can add /DEIGEN_STRONG_INLINE=inline in
CMAKE_CXX_FLAGS
3 - "ctest -j 9" won't speedup the builds, but only the run of the tests.
cmake . -DCMAKE_CXX_FLAGS="/MP4 /DEIGEN_STRONG_INLINE=inline"

Post by Patrik Huber
I've essentially followed your instructions and then
cmake .. -G "Visual Studio 15 2017 Win64"
ctest -C Release -D Experimental -j 9

Post by Gael Guennebaud
ctest -D Experimental

So that means no "-C Release"? What will that run?

all tests will be run instead of the ones in eigen/test only (in theory,
here none were run, which is why I recommended to remove this option).
In case of problems, you can rerun the tests without rebuilding binaries
ctest -D Experimental --build-noclean -VV
-VV is useful to see what's going on...
thanks again.

Post by Patrik Huber
I'm happy to let them run & finish or restart them with modified
commands, just let me know.
Kind regards,
Patrik
--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom
Web: www.patrikhuber.ch
Mobile: [phone] <+44%207482%20633934>

On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud <

Post by Gael Guennebaud
[...]
ctest -C Release -D Experimental

ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.
gael

--
Dr. Patrik Huber
Centre for Vision, Speech and Signal Processing
University of Surrey
Guildford, Surrey GU2 7XH
United Kingdom
Web: www.patrikhuber.ch
Mobile: [phone] <+44%207482%20633934>

[Edward Lam]

It's doesn't seem to be work for me on VS2017 though. I let it run for over 7
hours and it still hasn't finished on my i7-6850K 4.6 GHz with 64 MB of RAM, 6
cores, 12 threads on Windows 7. I've attached my build log, which was invoked
from the native 64-bit VS2017 cmd shell and generated using the windows versino
of cmake 3.7.2.

-Edward

Post by Gael Guennebaud
[...]
ctest -C Release -D Experimental
ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be welcome
to get feedback for MSVC 2015 and 2017.
gael

[Bill Greene]

I am also running the tests on 64-bit VS2017. My system is significantly
slower than yours and it took
over 16 hours to complete the tests.

Post by Edward Lam
It's doesn't seem to be work for me on VS2017 though. I let it run for
over 7 hours and it still hasn't finished on my i7-6850K 4.6 GHz with 64 MB
of RAM, 6 cores, 12 threads on Windows 7. I've attached my build log, which
was invoked from the native 64-bit VS2017 cmd shell and generated using the
windows versino of cmake 3.7.2.
-Edward

On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud <
[...]
ctest -C Release -D Experimental
ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.
gael

[Edward Lam]

I finally noticed that cmake was using the 32-bit cross compiler for 64-bit for
some reason even though I explicitly started the x64 Native Tools Command
Prompt. And then it looked like even using /MP4 didn't actually parallelize the
build because we're mostly building just 1 file per test executable.

Instead of trying to figure out how to get ctest to invoke msbuild with the
right parallelization option, I just went with the ninja generator instead. eg.
- Ensure ninja.exe is in the PATH (https://ninja-build.org/)
- cd eigen-3.3
- mkdir build
- cd build
- cmake -G Ninja ..
- ctest -D Experimental -j

That built in a few minutes(!) whereas before, I never managed to finish
building the tests in over 7 hours. The Ninja generator detected to use the
native x64 compiler (in the same command prompt window).

It's now been running about 2.5 hours and I'm test 285/772 now. So hopefully
it'll finish running the benchmark in another few hours.

-Edward

I am also running the tests on 64-bit VS2017. My system is significantly slower
than yours and it took
over 16 hours to complete the tests.
It's doesn't seem to be work for me on VS2017 though. I let it run for over
7 hours and it still hasn't finished on my i7-6850K 4.6 GHz with 64 MB of
RAM, 6 cores, 12 threads on Windows 7. I've attached my build log, which was
invoked from the native 64-bit VS2017 cmd shell and generated using the
windows versino of cmake 3.7.2.
-Edward
On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud
    [...]
    ctest -C Release -D Experimental
ctest -D Experimental
For the record, the build is green for msvc 11 (2012), it would still be
welcome to get feedback for MSVC 2015 and 2017.
gael

[Edward Lam]

Trying a test run with VS2015 in Release with -DEIGEN_TEST_CXX11=ON and found a
few reg tests taking over 4 hours to compile on an i7-6850K 6.6 GHz with 64 MB
RAM, 6 cores (12 threads) on Windows 7.

eg.

C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe /nologo /TP -Itest -I..\test -I..\
-I. /DWIN32 /D_WINDOWS /W4 /GR /EHsc /EHsc /wd4127 /wd4505 /wd4714
/D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /MD /O2 /Ob2 /DNDEBUG
-DEIGEN_TEST_MAX_SIZE=320 -DEIGEN_TEST_FUNC=sparse_product /bigobj
-DEIGEN_TEST_PART_2=1 /showIncludes
/Fotest\CMakeFiles\sparse_product_2.dir\sparse_product.cpp.obj
/Fdtest\CMakeFiles\sparse_product_2.dir\ /FS -c
C:\cygwin\tmp\eigen3.3\test\sparse_product.cpp
(currently using 2.77 GB)

C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe /nologo /TP -Iunsupported\test
-I..\unsupported\test -I..\ -I. -I..\unsupported\test\..\..\test
-I..\unsupported\test\..\..\unsupported -I..\unsupported\test\..\..\Eigen
-Iunsupported\test\..\..\test /DWIN32 /D_WINDOWS /W4 /GR /EHsc /EHsc /wd4127
/wd4505 /wd4714 /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /MD /O2 /Ob2
/DNDEBUG -DEIGEN_TEST_MAX_SIZE=320 -DEIGEN_TEST_FUNC=sparse_extra /bigobj
-DEIGEN_TEST_PART_2=1 /showIncludes
/Founsupported\test\CMakeFiles\sparse_extra_2.dir\sparse_extra.cpp.obj
/Fdunsupported\test\CMakeFiles\sparse_extra_2.dir\ /FS -c
C:\cygwin\tmp\eigen3.3\unsupported\test\sparse_extra.cpp
(currently using 2.59 GB)

There was more, but they seem to have finished in the time I got distracted and
came back an hour later.

-Edward

Post by Edward Lam
I finally noticed that cmake was using the 32-bit cross compiler for 64-bit for
some reason even though I explicitly started the x64 Native Tools Command
Prompt. And then it looked like even using /MP4 didn't actually parallelize the
build because we're mostly building just 1 file per test executable.
Instead of trying to figure out how to get ctest to invoke msbuild with the
right parallelization option, I just went with the ninja generator instead. eg.
- Ensure ninja.exe is in the PATH (https://ninja-build.org/)
- cd eigen-3.3
- mkdir build
- cd build
- cmake -G Ninja ..
- ctest -D Experimental -j
That built in a few minutes(!) whereas before, I never managed to finish
building the tests in over 7 hours. The Ninja generator detected to use the
native x64 compiler (in the same command prompt window).
It's now been running about 2.5 hours and I'm test 285/772 now. So hopefully
it'll finish running the benchmark in another few hours.
-Edward

Post by Bill Greene
I am also running the tests on 64-bit VS2017. My system is significantly
slower than yours and it took
over 16 hours to complete the tests.
    It's doesn't seem to be work for me on VS2017 though. I let it run for over
    7 hours and it still hasn't finished on my i7-6850K 4.6 GHz with 64 MB of
    RAM, 6 cores, 12 threads on Windows 7. I've attached my build log, which was
    invoked from the native 64-bit VS2017 cmd shell and generated using the
    windows versino of cmake 3.7.2.
    -Edward
        On Fri, Dec 15, 2017 at 9:54 AM, Gael Guennebaud
             [...]
             ctest -C Release -D Experimental
        ctest -D Experimental
        For the record, the build is green for msvc 11 (2012), it would still be
        welcome to get feedback for MSVC 2015 and 2017.
        gael

[Gael Guennebaud]

Thanks a lot for the numerous builds. After filtering false negatives, I
see two issues:

1 - geo_alignedbox_9, geo_alignedbox_10, geo_alignedbox_11 are segfaulting
on bgreene-hp

-> the backtrace would be nice

2 - several errors with intel compiler (sweepy)

-> I'll check if I can reproduce them on Linux.


gael

Post by Gael Guennebaud
Hi everybody,
it would be extremely helpful if some of you could build and run our unit
tests on the 3.3 branch with MSVC compilers. Basically all you need to do
hg clone https://bitbucket.org/eigen/eigen eigen3.3
cd eigen3.3
hg up 3.3
mkdir build
cd build
cmake ..
ctest -C Release -D Experimental
The outcome will be visible to us there: http://manao.inria.fr/
CDash/index.php?project=Eigen3.3
There more details there: http://eigen.tuxfamily.org/index.php?title=Tests
thanks for your help,
gael
PS: For the long story, I used to have continuous builds with MSVC but all
my VMs have been deleted by an error of our sysadmin, and I struggle to
recreate them.

[Edward Lam]

FYI, I've rerun some tests using Release and -DEIGEN_TEST_CXX11=ON on VS2017 and
it failed some tests. Results were uploaded.

-Edward

Thanks a lot for the numerous builds. After filtering false negatives, I see two
1 - geo_alignedbox_9, geo_alignedbox_10, geo_alignedbox_11 are segfaulting
on bgreene-hp
-> the backtrace would be nice
2 - several errors with intel compiler (sweepy)
-> I'll check if I can reproduce them on Linux.
gael
Hi everybody,
it would be extremely helpful if some of you could build and run our unit
hg clone https://bitbucket.org/eigen/eigen
<https://bitbucket.org/eigen/eigen> eigen3.3
cd eigen3.3
hg up 3.3
mkdir build
cd build
cmake ..
ctest -C Release -D Experimental
http://manao.inria.fr/CDash/index.php?project=Eigen3.3
<http://manao.inria.fr/CDash/index.php?project=Eigen3.3>
There more details there: http://eigen.tuxfamily.org/index.php?title=Tests
<http://eigen.tuxfamily.org/index.php?title=Tests>
thanks for your help,
gael
PS: For the long story, I used to have continuous builds with MSVC but all
my VMs have been deleted by an error of our sysadmin, and I struggle to
recreate them.