Resource partitioner #2789

hkaiser · 2017-07-31T20:18:16Z

No description provided.

biddisco · 2017-07-31T20:22:04Z

This cannot be merged until I (or you @hkaiser) eliminates and fixes the commit marked "WIP: FIX THIS PROPERLY"

basic structure of the resource_partitioner class working

…ogy_info

Affinity_data as data members of resource partitioner and make them friends

Add resource partitioner's initialization step to hpx_init(...). Add get_resource_partitioner() function returning reference to one static instance of the resource partitioner

Adapt the initialization flow in hpx_init(...). Undo type-erasure of runtime and threadmanager. Type erase thread_pool. Compiles, execution not tested yet.

Modify constructors of threadmanager and threadpool accordingly. Remove affinity_data and topology from scheduler

cleanup and add more coments

- change data member of threadmanager - change constructor of threadmanager - change member functions of threadmanager which act on pools - make changes to threadmanager functions accordingly - change internal data structure of resource-partitioner from vector<std::size_t> to mask_type - add function in resource-partitioner checking for: - oversubscription (pu sharing) - empty default pool - compatibility between --hpx:threads and number of PUS assigned - make sure the default pool pool is the one that gets instantiated first by the threadmanager

- fix compilation errors in threadmanager - resource partitioner instantiates a default thread pool at construction - threadmanager::default_pool() returns a pointer to default_pool

biddisco · 2017-08-08T08:52:53Z

Could I please ask @KhalidHasanov to have a look through this branch and check if the work he contributed to the throttling policy has been accidentally deleted. If this is the case, then I will either edit the commit that deleted it and force push, or if @KhalidHasanov can provide a patch that reinstates anything missing, we can merge it back in. Thanks

hkaiser · 2017-08-08T11:29:25Z

@KhalidHasanov, @sithhell, @biddisco I think this branch allows to remove the throttling policy altogether as it now can be held outside of the core HPX libraries. This scheduler is very project specific, so let's leave it in project space.

biddisco · 2017-08-08T11:54:57Z

Very good point. @KhalidHasanov you should have a look at the file

examples/resource_partitioner/simple_resource_partitioner.cpp

on this branch (around line 277) to see how to supply a custom scheduler to the default pool.
This allows you to package your scheduler in user code, rather than making it part of the main hpx distribution. At some point, we will probably go through the schedulers that are currently available and remove or clean up some of them.

KhalidHasanov · 2017-08-08T12:06:32Z

@biddisco Thanks for pointing out the example file. I will have a look once the branch is merged and see how it goes.

- fix --hpx:threads=all command line option to properly interact with hpx.os_threads=N configuration setting - adding more convenience functions for resource_partitioner - adding some convenience headers

- Reduced exposed user interface, changed creation of RP in user-code - shuffled things around, etc.

sithhell · 2017-08-22T06:22:59Z

examples/resource_partitioner/shared_priority_scheduler.hpp

+                victim_queues_.clear();
+                victim_queues_.resize(num_worker_queues);
+                //
+                BOOST_ASSERT(num_worker_queues != 0);


Can we switch to HPX_ASSERT here?

Good catch, thanks. We should add that to the obsolete macro tests in inspect.

sithhell · 2017-08-22T06:33:27Z

tests/unit/parallel/segmented_algorithms/partitioned_vector_target.cpp

+ #    include <hpx/runtime/serialization/dynamic_bitset.hpp>
+ #  endif
+ #endif
+


Shouldn't this be moved in the header of the target to enable its serialization?

Yes, this should go away. It's a leftover from Shoshana's experiments...

sithhell · 2017-08-22T13:48:25Z

Only a few minor comments here. This PR is rather big and we should get it in ASAP, and keep improving it as we gain experience with the new feature.

hkaiser · 2017-08-22T14:22:57Z

This PR is rather big and we should get it in ASAP, and keep improving it as we gain experience with the new feature.

I couldn't agree more.

hkaiser · 2017-08-23T19:31:28Z

@shoshijak Thanks again for all your work on this! @aserio will get in contact with you to send a STE||AR-Group t-shirt as a small token of appreciation of your work.

aserio · 2017-08-23T20:32:39Z

@shoshijak would you send me an email so we can discuss getting you a T-shirt?
adrian.serio@gmail.com

hkaiser added category: threadmanager type: enhancement labels Jul 31, 2017

hkaiser added this to the 1.1.0 milestone Jul 31, 2017

hkaiser force-pushed the resource_partitioner branch 9 times, most recently from f54f94c to 7a1dfbf Compare August 4, 2017 23:28

shoshijak added 17 commits August 5, 2017 17:49

Initial commit: add resource_partitioner directory and files to hpx

26b46f1

basic structure of the resource_partitioner class working

pre-hpx-main functionalities complete

08efe29

move files to runtime/resource

8b84167

extract .cpp content from .hpp

eee1b47

fix: getting global pointer and initialization of runtime

8df46ae

Add get_resource_partitioner_ptr

cdecb68

solve cyclic include, make resource_partitioner friend of hwloc_topol…

cca1821

…ogy_info

add scheduling policy to init_pool_data

9b37640

Rename init_pool_data and add init_affinity data

4875322

Affinity_data as data members of resource partitioner and make them friends

Start getting the resource partitioner into the runtime's hair.

7fcb3c5

Add resource partitioner's initialization step to hpx_init(...). Add get_resource_partitioner() function returning reference to one static instance of the resource partitioner

cleanup and add special functions for default pool

3b9c734

test default behaviors: ok

3cc56f9

integrate resource partitioner to web of classes

fe28711

Adapt the initialization flow in hpx_init(...). Undo type-erasure of runtime and threadmanager. Type erase thread_pool. Compiles, execution not tested yet.

Make threadmanager hold a pointer to scheduler

8dcfee7

Modify constructors of threadmanager and threadpool accordingly. Remove affinity_data and topology from scheduler

solve scope issues in constructor of threadmanager

faa296e

cleanup and add more coments

More fixes and enhancements to threadmanager and resource-partitioner

90d1d7a

- fix compilation errors in threadmanager - resource partitioner instantiates a default thread pool at construction - threadmanager::default_pool() returns a pointer to default_pool

hkaiser force-pushed the resource_partitioner branch from 1242ec2 to 45d3cff Compare August 8, 2017 00:24

More inspect happyness

5a05907

hkaiser added 2 commits August 8, 2017 12:01

Adding basic test for resource_partitioner

466cc16

- fix --hpx:threads=all command line option to properly interact with hpx.os_threads=N configuration setting - adding more convenience functions for resource_partitioner - adding some convenience headers

Refactoring resource partitioner code

03964cf

- Reduced exposed user interface, changed creation of RP in user-code - shuffled things around, etc.

hkaiser force-pushed the resource_partitioner branch from a5296f2 to 03964cf Compare August 10, 2017 17:21

hkaiser added 2 commits August 10, 2017 12:47

Merge branch 'master' into resource_partitioner

d5753a3

Fix exported symbols in resource::partition

9b97602

hkaiser force-pushed the resource_partitioner branch from d56eeeb to 9b97602 Compare August 10, 2017 23:01

Appeasing inspect

ea4070c

hkaiser force-pushed the resource_partitioner branch from 940e9bf to a9d2f7b Compare August 14, 2017 17:17

Fix exception handling, fix more than one locality/node

df54dd7

hkaiser force-pushed the resource_partitioner branch from a9d2f7b to df54dd7 Compare August 16, 2017 20:15

hkaiser added 2 commits August 16, 2017 18:04

Merge branch 'master' into resource_partitioner

bdafb76

Adapting to changes needed after merge

0bb901b

sithhell reviewed Aug 22, 2017

View reviewed changes

Addressing review comments

78b4f27

hkaiser force-pushed the resource_partitioner branch from a051f7e to 78b4f27 Compare August 22, 2017 20:46

sithhell approved these changes Aug 23, 2017

View reviewed changes

hkaiser merged commit 6c214b2 into master Aug 23, 2017

hkaiser deleted the resource_partitioner branch August 23, 2017 19:30

sithhell mentioned this pull request Sep 12, 2017

Change default number of threads used to the maximum of the system #2324

Closed

biddisco added this to completed in resource partitioner/manager work Oct 12, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Resource partitioner #2789

Resource partitioner #2789

hkaiser commented Jul 31, 2017

biddisco commented Jul 31, 2017

biddisco commented Aug 8, 2017

hkaiser commented Aug 8, 2017

biddisco commented Aug 8, 2017

KhalidHasanov commented Aug 8, 2017

sithhell Aug 22, 2017

hkaiser Aug 22, 2017

sithhell Aug 22, 2017

hkaiser Aug 22, 2017

sithhell commented Aug 22, 2017

hkaiser commented Aug 22, 2017

hkaiser commented Aug 23, 2017

aserio commented Aug 23, 2017

Resource partitioner #2789

Resource partitioner #2789

Conversation

hkaiser commented Jul 31, 2017

biddisco commented Jul 31, 2017

biddisco commented Aug 8, 2017

hkaiser commented Aug 8, 2017

biddisco commented Aug 8, 2017

KhalidHasanov commented Aug 8, 2017

sithhell Aug 22, 2017

Choose a reason for hiding this comment

hkaiser Aug 22, 2017

Choose a reason for hiding this comment

sithhell Aug 22, 2017

Choose a reason for hiding this comment

hkaiser Aug 22, 2017

Choose a reason for hiding this comment

sithhell commented Aug 22, 2017

hkaiser commented Aug 22, 2017

hkaiser commented Aug 23, 2017

aserio commented Aug 23, 2017