The use of D-Wave on AWS is hampered by library size.

Not sure posting this here makes sense as the age of comments makes it seem as this isn't where the community goes for answers.  If there is a better place please add a comment.

I have posted this on the AWS Braket Forum as well.

Folks,
I have put together a Lambda function that does a very basic quantum calculation that works fine when I run it locally in the SAM SDK. When I go to deploy it the deploy fails because the total unzipped size of the deployment package is greater than 250MB. I think it's around 264MB.

As the deployment size quota is fixed and can't be changed.

Any ideas on how to fix this? Is there a small set of libraries that will fit?

In looking at the dependencies below there may be a need for some library refactoring to get this to the point that it can be used in a Lambda function.


The larger packages are (look below for full list):
{code}
48M botocore
112K botocore-1.19.0.dist-info
20M dimod
36K dimod-0.9.5.dist-info
21M minorminer
28K minorminer-0.2.3.dist-info
17M numpy
112K numpy-1.19.2.dist-info
31M numpy.libs
29M pydantic
92K pydantic-1.6.1.dist-info
49M scipy. Looks like it uses two routines optimize and spatial out of spicy
172K scipy-1.5.3.dist-info
29M scipy.libs
{code}

In the code this what I am using.
{code}
from braket.aws import AwsDevice
from braket.ocean_plugin import BraketDWaveSampler
from dimod.binary_quadratic_model import BinaryQuadraticModel
from dimod import BINARY
from dwave.system.composites import EmbeddingComposite
{code}

I have gone through and stripped as many of the libraries as is possible in the requirements.txt file but the size is still to big.
{code}
amazon-braket-ocean-plugin==1.0.0.post1
dwave-system
dimod==0.9.5
{code}

These packages have these dependencies:
{code}
amazon-braket-ocean-plugin==1.0.1.post2
- amazon-braket-sdk [required: Any, installed: 1.1.2.post1]
- amazon-braket-default-simulator [required: Any, installed: 1.0.0.post2]
- amazon-braket-schemas [required: Any, installed: 1.0.2.post1]
- pydantic [required: Any, installed: 1.6.1] ==> Large 29M
- numpy [required: Any, installed: 1.19.2]. ==> Large 48M
- opt-einsum [required: Any, installed: 3.3.0]
- numpy [required: >=1.7, installed: 1.19.2] ==> Large 48M
- backoff [required: Any, installed: 1.10.0]
- boltons [required: Any, installed: 20.2.1]
- boto3 [required: Any, installed: 1.16.0]
- botocore [required: >=1.19.0,<1.20.0, installed: 1.19.0] ==> Large 48M
- jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
- python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- urllib3 [required: >=1.25.4,<1.26, installed: 1.25.11]
- jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
- s3transfer [required: >=0.3.0,<0.4.0, installed: 0.3.3]
- botocore [required: >=1.12.36,<2.0a.0, installed: 1.19.0]
- jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
- python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- urllib3 [required: >=1.25.4,<1.26, installed: 1.25.11]
- nest-asyncio [required: Any, installed: 1.4.1]
- networkx [required: Any, installed: 2.5]
- decorator [required: >=4.3.0, installed: 4.4.2]
- numpy [required: Any, installed: 1.19.2]. ==> Large 48M
- boltons [required: Any, installed: 20.2.1]
- boto3 [required: Any, installed: 1.16.0]
- botocore [required: >=1.19.0,<1.20.0, installed: 1.19.0]
- jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
- python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- urllib3 [required: >=1.25.4,<1.26, installed: 1.25.11]
- jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
- s3transfer [required: >=0.3.0,<0.4.0, installed: 0.3.3]
- botocore [required: >=1.12.36,<2.0a.0, installed: 1.19.0]
- jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
- python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- urllib3 [required: >=1.25.4,<1.26, installed: 1.25.11]
- dimod [required: >=0.8.13, installed: 0.9.10] ==> Large 20M
- numpy [required: >=1.16.0,<2.0.0, installed: 1.19.2] ==> Large 48M
dwave-system==1.2.1
- dimod [required: >=0.9.7,<0.10.0, installed: 0.9.10] ==> Large 20M
- numpy [required: >=1.16.0,<2.0.0, installed: 1.19.2] ==> Large 48M
- dwave-cloud-client [required: >=0.8.1,<0.9.0, installed: 0.8.1]
- click [required: >=7.0, installed: 7.1.2]
- homebase [required: >=1.0, installed: 1.0.1]
- plucky [required: >=0.4.3, installed: 0.4.3]
- python-dateutil [required: >=2.7, installed: 2.8.1]
- six [required: >=1.5, installed: 1.15.0]
- requests [required: >=2.18, installed: 2.24.0]
- certifi [required: >=2017.4.17, installed: 2020.6.20]
- chardet [required: >=3.0.2,<4, installed: 3.0.4]
- idna [required: >=2.5,<3, installed: 2.10]
- urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.11]
- dwave-networkx [required: >=0.8.4, installed: 0.8.8]
- decorator [required: >=4.1.0,<5.0.0, installed: 4.4.2]
- dimod [required: >=0.8.0, installed: 0.9.10] ==> Large 20M
- numpy [required: >=1.16.0,<2.0.0, installed: 1.19.2] ==> Large 48M
- networkx [required: >=2.0,<3.0, installed: 2.5]
- decorator [required: >=4.3.0, installed: 4.4.2]
- dwave-tabu [required: >=0.2.0, installed: 0.3.0]
- dimod [required: >=0.9.0, installed: 0.9.10] ==> Large 20M
- numpy [required: >=1.16.0,<2.0.0, installed: 1.19.2] ==> Large 48M
- numpy [required: >=1.16, installed: 1.19.2] ==> Large 48M
- homebase [required: >=1.0.0,<2.0.0, installed: 1.0.1]
- minorminer [required: >=0.2.0,<0.3.0, installed: 0.2.3] ==> Large 21M
- dwave-networkx [required: Any, installed: 0.8.8]
- decorator [required: >=4.1.0,<5.0.0, installed: 4.4.2]
- dimod [required: >=0.8.0, installed: 0.9.10] ==> Large 20M
- numpy [required: >=1.16.0,<2.0.0, installed: 1.19.2] ==> Large 48M
- networkx [required: >=2.0,<3.0, installed: 2.5]
- decorator [required: >=4.3.0, installed: 4.4.2]
- fasteners [required: Any, installed: 0.15]
- monotonic [required: >=0.1, installed: 1.5]
- six [required: Any, installed: 1.15.0]
- homebase [required: Any, installed: 1.0.1]
- networkx [required: Any, installed: 2.5]
- decorator [required: >=4.3.0, installed: 4.4.2]
- numpy [required: Any, installed: 1.19.2] ==> Large 48M
- scipy [required: Any, installed: 1.5.3] ==> Huge 78M
- numpy [required: >=1.14.5, installed: 1.19.2] ==> Large 48M
- networkx [required: >=2.0,<3.0, installed: 2.5]
- decorator [required: >=4.3.0, installed: 4.4.2]
- numpy [required: >=1.14.0,<2.0.0, installed: 1.19.2] ==> Large 48M
- six [required: >=1.11.0,<2.0.0, installed: 1.15.0]
pipdeptree==1.0.0
- pip [required: >=6.0.0, installed: 20.1.1]
PySocks==1.7.1
setuptools==47.1.0
{code}

In examining the .aws-sam/build directory there may be library elements in there that are not necessary.
{code}
32K PySocks-1.7.1.dist-info
0B __init__.py
40K amazon_braket_default_simulator-1.0.0.post2.dist-info
36K amazon_braket_ocean_plugin-1.0.0.post1.dist-info
40K amazon_braket_schemas-1.0.2.post1.dist-info
40K amazon_braket_sdk-1.1.2.post1.dist-info
8.0K app.py
36K backoff
28K backoff-1.10.0.dist-info
600K boltons
20K boltons-20.2.1.dist-info
1.0M boto3
36K boto3-1.16.0.dist-info
48M botocore
112K botocore-1.19.0.dist-info
696K braket
288K certifi
20K certifi-2020.6.20.dist-info
444K chardet
28K chardet-3.0.4.dist-info
304K click
20K click-7.1.2.dist-info
468K dateutil
28K decorator-4.4.2.dist-info
20K decorator.py
20M dimod
36K dimod-0.9.5.dist-info
680K dwave
36K dwave_cloud_client-0.8.1.dist-info
324K dwave_networkx
28K dwave_networkx-0.8.8.dist-info
32K dwave_system-1.2.1.dist-info
28K dwave_tabu-0.3.0.dist-info
80K fasteners
28K fasteners-0.15.dist-info
28K homebase
16K homebase-1.0.1.dist-info
276K idna
28K idna-2.10.dist-info
80K jmespath
4.0K jmespath-0.10.0.data
36K jmespath-0.10.0.dist-info
12K makenumbers.py
21M minorminer
28K minorminer-0.2.3.dist-info
24K monotonic-1.5.dist-info
8.0K monotonic.py
20K nest_asyncio-1.4.1.dist-info
8.0K nest_asyncio.py
5.7M networkx
488K networkx-2.5.data
88K networkx-2.5.dist-info
17M numpy
112K numpy-1.19.2.dist-info
31M numpy.libs
256K opt_einsum
28K opt_einsum-3.3.0.dist-info
24K plucky
28K plucky-0.4.3.dist-info
29M pydantic
92K pydantic-1.6.1.dist-info
28K python_dateutil-2.8.1.dist-info
216K requests
36K requests-2.24.0.dist-info
4.0K requirements.txt
272K s3transfer
24K s3transfer-0.3.3.dist-info
49M scipy
172K scipy-1.5.3.dist-info
29M scipy.libs
20K six-1.15.0.dist-info
36K six.py
32K socks.py
4.0K sockshandler.py
2.8M tabu
40K test
412K urllib3
60K urllib3-1.25.11.dist-info
{code}

0

Comments

1 comment

Please sign in to leave a comment.

Didn't find what you were looking for?

New post