MAIL-IN/HAND-IN User Survey

We have been operating our SAXS mail-in/hand-in program for 6 months – 3 cycles of ALS proposals. The beamline and its staff are dedicated to developing and applying technologies and methodologies which combine SAXS and crystallography. Interacting with an extensive and robust external user community is a critical component of this development. Our mission for the mail-in/hand-in program is to provide SAXS data to users at as high or higher quality than if users were to come to the beamline and collect these data sets themselves. We believe this is possible since the data collection is now fast and the beamline interface has grown necessarily complex. Beamline staff which develop the interface are more likely to optimize data collection and spot problems earlier than most users. We initiated this program as we were no longer able to meet the growing demand for beamtime with all users coming on-site for data collection. Most users who participated in this program would not have been able to collect data at SIBYLS were it not for the mail-in/hand-in program.

We have conducted a survey of labs that have used the modality of SIBYLS to provide feedback – which we take very seriously. This should help inform new interested labs on how the system works. In addition we will compare input from future surveys against this one as a measure of progress. The number of labs which have gained access to SAXS data collection at SIBYLS has increased 10-fold.

We have significant room to grow and improve the system as highlighted by several users in our survey. Aside from developing the infrastructure for trouble free collection, a challenge is balancing the most easily implemented one size fits all approach with some user adjustablility. We hope to incorporate more flexibility for users as we develop. While the system has not been entirely trouble free – given the programs novelty, we are pleased with the feedback and encouraged that the mail-in/hand-in program is usefully continued. For example most users indicated they would use the mail-in/hand-in system again and a substantial number have papers in preparation based on data collected at the beamline. The survey and its results can be viewed by clicking the link below.

27 Labs provided Survey Responses

Question 1

Comments:

• Actually, we have two papers containing the SAXS data at SIBYLS. One is just accepted, the other is under minor revision.
• Since we are local we prefer to perform experiments on our own samples. We have published a paper using SAXS data acquired by us at SIBYLS and have a manuscript in prep. It is required to optimize exposure times and sample concentrations on the fly and we feel most comfortable doing this ourselves because the samples are sensitive.
• The mail-in SAXS service really helps our research. The fact that we don’t have to travel or do data collection our selves it a great time saver. Also the way the data is presented is very easy and intuitive. I have been to other places where data collection and analysis is much (much) more troublesome. Great job!
• Working with Kevin was great, but unfortunately SAXS did not help me, because of the behavior of my protein.
• I would probably know what to do with the data if it were from a well-behaved model system like lysozyme, but my stuff is more complicated, so I don’t yet know if my data are useful.
• For simple, aqueous, samples I think that the current set-up is very convenient; accessing data etc is easy. Our last batch of samples (mail in) contained PEG MW 8000. We anticipated no problem with these samples, particularly with your washing protocol. In the event, quite a few of the samples contained bubbles (the image files recorded of the sample chamber were useful in diagnosing this). Additionally, even those without bubbles seemed to experience some PEG aggregation. This would, of course, have happened with us at the beam. But we would have monitored this, and stopped it when the data looked unusual…..
• My experience is that sending SAXS samples is not necessarily reliable. The combination of the samples being sensitive (not frozen), very expensive to prepare, and the fact that FedEx cannot be trusted to deliver as promised has made us rethink our approach to SAXS. We will consider hand-delivery across the country before using mail-in again.
• We have collected all types of samples and have all types of quality data. For the no-brainers, its easy and straight forward. For the remaining 85% of the samples its difficult and being able to talk to someone will be imperative if you want this to grow. The mailin is very helpful…………but I just love coming to Berkley periodically, which does help in understanding how the beamline can be used to improve our experiments.

Question 2

Comments:

• raw data
• I found some of the preliminary analysis a bit too stringent. “Radiation sensitive” or “aggregation” gives the impression that the data is not good, when in fact it can still be valuable. Maybe soften to “shows sign of radiation damage/aggregation’
• Had issues with background subtraction that required going back to the original data.
• Currently, your data processing makes it hard to export the background and sample scattering curves as .dat files for viewing/analysis in a generic program. I asked for a scattering curve recorded without the beam on so that I could subtract this from the background and sample curves separately, effectively being able to view the curves before subtraction. I found this to be extremely useful.
• I need to better understand what exactly happens when data images are processed, what detector characteristics are etc. I need this information because I wish to better judge the information content of my data. It appears that this information is not published or otherwise available.
• In general, I’ve been happy with the data files and the preliminary analysis. In case reliable and automated data analysis programs exist, it would be good to get Rg/P(r), …etc as well.
• Everything! The more information the better. We wanted to compare The preliminary analysis was good, but it would be nice to be able to better understand the comments. For example, WHY is the data flagged by radiation damage? Maybe some type of additional file can be added to help users understand what exactly to look for in their data. Additional processing might be helpful as a teaching tool. If an expert processes 1 dataset, then users might try to reproduce this processing. This cannot be automatic, or it has to be checked to makre sure that it is correct. Also, it would be nice to have several standard proteins that are always collected on the line. This would provide a nice calibration and might also be used to help new users understand the process.
• I needed the primary data to do some alternative buffer subtractions. Also, although we process our data to get Rg and Pr, etc, it would be great to have a preliminary assessment of data quality provided by the beamline. As you know, it can be somewhat labor intensive to do the assessment of 96 or more samples. Getting an idea of which samples are worthy of further analysis would help.
• Not clear what you mean by primary data. I found the preliminary analysis very useful but still preferred to have access to the primary files for further analysis.

Question 3

Comments:

• We run all our samples over a gel filtration column and freeze them directly before shipping. Still many of our samples show some degree of aggregation, which is likely be be caused by the freezing and thawing. Also, filling a 96 well plate can take several hours which means that not all samples are equally ‘fresh’. If it would be possible to send the samples in PCR-strips (with 8 vials in a strip that would then fit into a pre-shaped holder) that could improve the samples. Alternatively, if it would be possible to have a gelfiltration column in-line with the sample cell (i.e. similar to SEC-MALS) that would greatly improve the quality of the data enormously. Plus it will give information about the stability of complexes as weaker complexes will separate on the column, giving rise to different scattering profiles.
• We might like to specify exposure times for some samples.
• I sent additional instructions for pre-experimental sample heating/vortexing/centrifuging.
• I would sacrafice being able to send more plates, if it meant I could have more interaction with the staff.
• I would like the flexibility of preparing samples myself, instead of letting them sit for an indefinite period of time.
• The sample guidelines suggest 25ul per sample. We encountered a couple of samples that were too low in volume. We now send larger volumes to prevent this from happening.

Question 4

Comments:

• We do all these experiments at home and prefer continue operating in this manner. For example, we routinely characterize our samples by SEC-MALLS prior to SAXS.
• Gel filtration column in-line with sampe cell (similar to SEC-MALS)
• A gel filtration chromatography setup (GF-HPLC) was coupled with SAXS
• Gel Filtration
• We have found it very useful to do SEC-MALS prior to SAXS to know the MW of our samples

Question 5

Comments:

• If time resolved SAXS were possible (with a stopped flow device or similar), several people in the lab would have immediate experiments that we’d like to try.
• It would be nice to use SAXS envelopes for phasing crystallographic data. I believe larger q is require for this to work. I don’t know whether 0.6 is high enough

Question 6

Comments:

• SAXS data on our samples seem a little less powerful for conformational change (option 3), but are good enough for option 1 and 2.

What is the most important change we can make to the system from your perspective?

• I have found my experience using the robotics to be great. Like mentioned above, getting access to lower q range would be helpful, and setting up a system for time-resolved studies would also be very nice. Thanks.
• It works very well but sometimes the baseline of scattering was unstable.
• Overall, the mailin SAXS you offer is great. Key for my current research is the ability to measure data from quite dilute samples.
• You guys have the best SAXS beamline that we’ve tried. It really is stellar. We also appreciate your drive to push the envelope of the beamline’s capabilities. We would like time resolved SAXS if possible. Perhaps also a gel filtration column where fractions can be directly hit with xrays upon coming off the column. Thanks!
• Being able to send a smaller number of samples (eg. a row of a plate instead of a whole plate) and sending them more frequently would be very helpful if there was some way to make it work with the system. Also, we have mostly collected data at the beamline where the collection and analysis seems to require a lot of manual steps. Any automation to the data collection and analysis pipeline would be very helpful as well.
• With the risk of repeating myself: I believe that having a gel-filtration column inline with the sample cell will be a great improvement. Although the data collection time would be a bit slower (a typical run takes 30 minutes), the quality of the data improves (no more aggregates in the sample), a dilution series is made by the column, and for complexes it can be found out if you have one unique species or a mixture. Specifically the last item is important as it is crucial that the sample is mono-disperse. But finding out if the sample is a mixture or not is not straightforward, so if it can be done during the SAXS experiment it would be extremely helpful. So while you might loose somewhat in data quantity, you gain a lot in data quality, which is what we all want in the end.
• If you dont know how to interpret SAXS data, you get stuck. So more help in that area would be great. Kevin helped me with my data and he was awesome!!! I just wish my protein behaved in the SAXS tube.
• You need to have a robust on-line tutorial with descriptions of what diagnstics to use, etc. when processing the data. Lists of data formats that are input and output by the various programs (including the data that the user receives from the beamline), recommended programs to use based on the experts’ experience for various scenarios would be It’s hard to know where to start when, e.g. Gunier gives you a garbage Rg and GNOM gives you something that is entirely consistent with what is known from structural studies. Some issues will clearly require expert intervention and/or posting to the message board, but other issues should be common enough to warrant a tutorial/wiki/faq.
• I think that more technical support would be appreciated, especially if it is clear that a group is not set up for data processing
• Experience was pretty close to optimal.
• An online form where I could specify how each sample would be collected (like 10x 1sec or 5x 5s). If the web page would generate a script it should not requite any human attantion.
• Effective communication is the number one change. It is currently too difficult to get information including the following: when is my (mail in) SAXS time scheduled? Have many sample and how much of each sample should I send? When will my samples be processed? How do I get the maximum amount of information out of my experiments? This is closely followed and related to the proposal process. It is completely unclear what the criteria are for a proposal and it is completely impossible to get feedback. The staff handling proposals are 100% unresponsive, it does not matter how many times or how you ask for clarification. This latter item is eventually going to kill the program, there are other places where this science can be done!
• more beam time.
• I can’t think of…you guys have done very nice job…maybe a routine sample submission schedule (like every the other Wednesday).
• If I could get better, or equivalent, data with less sample that would be the most important. Second, I have tried to use the processing programs on the SYB computer cluster. However, I am unable to get them to work. Eg. things are submitted, but never returned to me? This could be very, VERY, helpful if this was made user friendly and robust.
• I think this is a great service provided to the structural biology community! I’m particularly interested in a lower Q and lower concentration measurements for the study of large macromolecular assemblies purified from endogenous sources (ie: Ribosomes, transcriptional complexes). Thank you! Thanks to data obtained from this service over two sessions, we have three manuscripts in preparation that utilize this analysis.
• Rapid access would be nice. For example, we currently have a case in which I want to re-run a few samples (less than 10) in order to double-check results for a paper I am writing. It would be nice to have quick access via the mail-in program for these cases. Maybe this is already possible? Anaerobic data collection would be nice.
• Install a GF-MALS system to run the samples right before SXS analysis. It would be great if we could have a rapid access to send more plates with fewer samples. Currently we have once data collection for a 96 well plate per cycle. This is wonderful and useful. However, optimizing samples is not possible. Thus, if we send a sample that does not survive the shipping or is suboptimal due to buffer conditions we cannot quickly optimize to send a second sample. Any method to help improve access would be appreciated.
• This is actually an informative survey. Good job!