[Bio-Linux] Problem with adding data to GALAXY using FTP upload via transfer directory

Fri Jan 16 06:12:38 EST 2015

Hi Ian,

The correct permission is:

drwxrwx--- 2 user2  Debian-galaxy 4096 Feb 26  2014 user2 at localhost
(or in your case, manager rather than user2)

And anything else is going to cause problems.  Galaxy on Bio-Linux is
supposed to sort out the permissions when it starts up, but I had a
couple of bugs in the old version of that startup script that could
leave directories stuck with the wrong permissions.  Would you mind
trying deleting the manager at localhost directory entirely then restarting
Galaxy.  What permissions does it get recreated with?

Cheers,

TIM

On Fri, 2015-01-16 at 10:45 +0000, Ian Donaldson wrote:
> Hi,
> 
> I have followed the advice given in
> http://environmentalomics.org/bio-linux-galaxy/ and I have
> successfully set up the galaxy-server-apache-proxy.  I have put a sam
> file (ex1.sam) in the transfer directory for manager at localhost and can
> see the file in the'Files uploaded via FTP' section.  However, when I
> try to select and upload the file I get errors:
> 
> From the debug message:
> Traceback (most recent call last):
>   File "/usr/lib/galaxy-server/tools/data_source/upload.py", line 394,
> in <module>
>     __main__()
>   File "/usr/lib/galaxy-server/tools/data_source/upload.py", line 383,
> in __main__
>     add_file( dataset, registry, json_file, output_path )
>   File "/usr/lib/galaxy-server/tools/data_source/upload.py", line 278,
> in add_file
>     line_count, converted_path = sniff.convert_newlines( dataset.path,
> in_place=in_place )
>   File "/usr/lib/galaxy-server/lib/galaxy/datatypes/sniff.py", line
> 116, in convert_newlines
>     shutil.move( temp_name, fname )
>   File "/usr/lib/python2.7/shutil.py", line 302, in move
>     copy2(src, real_dst)
>   File "/usr/lib/python2.7/shutil.py", line 130, in copy2
>     copyfile(src, dst)
>   File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
>     with open(dst, 'wb') as fdst:
> IOError: [Errno 13] Permission denied:
> u'/var/lib/galaxy-server/transfer/manager at localhost/ex1.sam'
> 
> 
> From the server.log:
> galaxy.tools.actions.upload_common INFO 2015-01-16 10:35:07,359 tool
> upload1 created job id 2
> 127.0.0.1 - - [16/Jan/2015:10:35:07 +0000]
> "POST /galaxy/tool_runner/index HTTP/1.1" 200 -
> "http://localhost/galaxy/tool_runner?tool_id=upload1" "Mozilla/5.0
> (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> galaxy.jobs DEBUG 2015-01-16 10:35:07,485 (2) Working directory for
> job is: /usr/lib/galaxy-server/database/job_working_directory/000/2
> galaxy.jobs.handler DEBUG 2015-01-16 10:35:07,496 (2) Dispatching to
> local runner
> 127.0.0.1 - - [16/Jan/2015:10:35:07 +0000]
> "GET /galaxy/api/histories/1cd8e2f6b131e891/contents HTTP/1.1" 200 -
> "http://localhost/galaxy/root/index" "Mozilla/5.0 (X11; Ubuntu; Linux
> x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> galaxy.jobs DEBUG 2015-01-16 10:35:07,599 (2) Persisting job
> destination (destination id: local:///)
> galaxy.jobs.handler INFO 2015-01-16 10:35:07,622 (2) Job dispatched
> galaxy.tools.deps DEBUG 2015-01-16 10:35:07,770 Building dependency
> shell command for dependency 'samtools'
> galaxy.tools.deps WARNING 2015-01-16 10:35:07,770 Failed to resolve
> dependency on 'samtools', ignoring
> galaxy.jobs.runners DEBUG 2015-01-16 10:35:07,833 (2) command is:
> python /usr/lib/galaxy-server/tools/data_source/upload.py /usr/lib/galaxy-server /usr/lib/galaxy-server/database/tmp/tmp7LWfBs /usr/lib/galaxy-server/database/tmp/tmp9nLfry         2:/usr/lib/galaxy-server/database/job_working_directory/000/2/dataset_2_files:/usr/lib/galaxy-server/database/files/000/dataset_2.dat
> galaxy.jobs.runners.local DEBUG 2015-01-16 10:35:07,834 (2) executing
> job
> script: /usr/lib/galaxy-server/database/job_working_directory/000/2/galaxy_2.sh
> galaxy.jobs DEBUG 2015-01-16 10:35:07,862 (2) Persisting job
> destination (destination id: local:///)
> galaxy.jobs.runners.local DEBUG 2015-01-16 10:35:08,617 execution
> finished: /usr/lib/galaxy-server/database/job_working_directory/000/2/galaxy_2.sh
> galaxy.tools.deps DEBUG 2015-01-16 10:35:08,684 Building dependency
> shell command for dependency 'samtools'
> galaxy.tools.deps WARNING 2015-01-16 10:35:08,684 Failed to resolve
> dependency on 'samtools', ignoring
> galaxy.jobs.runners DEBUG 2015-01-16 10:35:08,684 executing external
> set_meta script for job
> 2: /usr/lib/galaxy-server/set_metadata.sh ./database/files /usr/lib/galaxy-server/database/job_working_directory/000/2 . /tmp/universe_wsgi_jymQ.ini /usr/lib/galaxy-server/database/tmp/tmp7LWfBs /usr/lib/galaxy-server/database/job_working_directory/000/2/galaxy.json /usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_in_HistoryDatasetAssociation_2_eqLSN4,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_kwds_HistoryDatasetAssociation_2_Y8KWCA,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_out_HistoryDatasetAssociation_2_e8sRbx,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_results_HistoryDatasetAssociation_2_s439LW,,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_override_HistoryDatasetAssociation_2_jlOhII
> galaxy.jobs.runners DEBUG 2015-01-16 10:35:09,731 execution of
> external set_meta for job 2 finished
> galaxy.jobs DEBUG 2015-01-16 10:35:09,789 setting dataset state to
> ERROR
> galaxy.jobs DEBUG 2015-01-16 10:35:09,874 job 2 ended
> galaxy.datatypes.metadata DEBUG 2015-01-16 10:35:09,874 Cleaning up
> external metadata files
> 127.0.0.1 - - [16/Jan/2015:10:35:11 +0000]
> "GET /galaxy/api/histories/1cd8e2f6b131e891/contents HTTP/1.1" 200 -
> "http://localhost/galaxy/root/index" "Mozilla/5.0 (X11; Ubuntu; Linux
> x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> 127.0.0.1 - - [16/Jan/2015:10:35:11 +0000]
> "GET /galaxy/api/histories/1cd8e2f6b131e891 HTTP/1.1" 200 -
> "http://localhost/galaxy/root/index" "Mozilla/5.0 (X11; Ubuntu; Linux
> x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> 
> 
> I noticed permission being mentioned.  The status of the transfer
> directory is:
> drwxr-xr-x 2 root root 4096 Jan 16 10:12 manager at localhost
> 
> I change this to:
> drwxr-xr-x 2 manager manager 4096 Jan 16 10:12 manager at localhost
> 
> But the same thing happened.
> 
> Has anyone seen this problem before and have and guidance?
> 
> Thank you,
> Ian
> 
> 

-- 
Tim Booth <tbooth at ceh.ac.uk>
NERC Environmental Bioinformatics Centre 

Centre for Ecology and Hydrology
Maclean Bldg, Benson Lane
Crowmarsh Gifford
Wallingford, England
OX10 8BB 

http://environmentalomics.org/bio-linux
+44 1491 69 2297