Output of LeapHybrid takes long to evaluate

Hello together, 

we made the strange observation that the evaluation of the output in form of the dimod.SampleSet takes a long time to evaluate especially in comparison to the actual sampling step. Here is a minimal code example 

import networkx as nx
import dimod

K_7 = nx.complete_graph(7)
bqm = dimod.generators.random.ran_r(1, K_7)

start = time.time()
hybrid_solver = LeapHybridSampler(solver = None, connection_close = False, token = TOKEN)
print('\nDuration init hybrid sampler', time.time() - start)

start = time.time()
sample_set = hybrid_solver.sample(bqm, time_limit=TIME_LIMIT_CLASSIC_SOLVER)
print('\nDuration get sample', time.time() - start)

start_time = time.time()
print(sample_set)
print('\n Duration print sample set', time.time() - start_time)

With output: 

Duration init hybrid sampler 2.8732540607452393

Duration get sample 2.669390916824341
   0  1  2  3  4  5  6 energy num_oc.
0 -1 +1 +1 -1 -1 +1 +1   -9.0       1
['SPIN', 1 rows, 1 samples, 7 variables]

 Duration print sample set 13.532737016677856

 

This means that the evaluation of the print statement right after the sampling of the hybrid solver is done, takes a long time to evaluate, much longer than the actual init and sampling step. We further made the observation, that if I print the sample set in another cell after waiting for roughly 13 seconds, the print statement is almost immediately evaluated. I also switched 'connection_close' from True to False and vice versa but nothing changed. 

This makes me guess that there is still some sort of connection of the hybrid sampler to the backend ongoing which first needs to be closed before the sample set can be evaluated/printed locally. 

Any ideas on how to fix this? 

Thanks in advance and best regards,

Simon

0

Comments

4 comments
  • Hello,

    Some components are asynchronous for efficiency. 

    You can run the above with the following to force the SampleSet to resolve before moving on to the print statement:

    start = time.time()
    sample_set = hybrid_solver.sample(bqm, time_limit=TIME_LIMIT_CLASSIC_SOLVER)
    sample_set.resolve()
    print('\nDuration get sample', time.time() - start)

    I hope this is helpful!

    Please let us know if you have any questions.

    0
    Comment actions Permalink
  • Hi David, 

    yes, that already helps. Thank you! 


    I suggest clarifying the documentation of the LeapHybrid Sampler https://docs.ocean.dwavesys.com/projects/system/en/stable/reference/samplers.html#dwave.system.samplers.LeapHybridSampler since this caused a lot of confusion for me and maybe for others as well. 

    Best,

    Simon

    0
    Comment actions Permalink
  • Thanks for the suggestion! 
    I have passed this along to the appropriate team. 
    Did you have any specific changes you would make that would help in your particular situation?

    The samplers have been written in such a way as to allow for synchronous and asynchronous (using futures) functionality under the hood to make them more adaptable and flexible.
    Bringing some more light to this information would be useful I think.

    Thank you again for your input and collaborative efforts!

    0
    Comment actions Permalink
  • Hey David, 

    nice to see that you are so open to suggestions and changes.

    Yes, I have a couple of things in mind: 

    1. First of all, it would be indeed nice to bring more light to the asynchronous behavior of the D-Wave and Leap Hybrid Sampler. For example, I had no clue what was going on and also didn't know about resolve() method. 

    2. An overview of the available methods, such as .resolve(), would be nice. I think the appropriate place would be here: https://docs.ocean.dwavesys.com/projects/system/en/stable/reference/samplers.html#dwave.system.samplers.LeapHybridSampler 

    3. Then, to be honest, I think the documentation could be better structured. For example, there are three complementary websites that each contain information to the leap hybrid solver but they do not refer to each other. Here: https://docs.dwavesys.com/docs/latest/doc_leap_hybrid.html#leap-s-hybrid-solvers , here: https://docs.ocean.dwavesys.com/en/stable/overview/hybrid.html#leap-s-hybrid-solvers and the link that I just posted above. I personally think that are more centralized and better-structured documentation would greatly help beginners. 

    4. I made another, at least for me counterintuitive, observation. When setting a time limit for the leap hybrid solver the solver behaved as it was more like a minimum run time than a maximum time. So when setting the time limit for example to 100 seconds, I would have expected that the solver runs at most for 100s but most of the time in less especially for small problem instances. However, it turned out that the solver runs always for 100seconds even for very small toy problems. 

    5. I know the following is hard to resolve because it is your trade secret and maybe your unique selling point, but the D-Wave hybrid solver is a black-box algorithm. This makes it difficult for companies who want to use your service to solve real-world problems and potentially sell the solution to other customers. I think, that if you really want that your service is commercially used, you should bring a little bit more light into the working mode of the hybrid solver. 

    I hope that these suggestions make sense to you. 

    Best,
    Simon 

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post