Project

General

Profile

Support #4933

Upload and host files locally

Added by Stephen Klein over 3 years ago. Updated over 1 year ago.

Status:
Abandoned
Priority name:
High
Assignee:
Category name:
-
Target version:
Start date:
2015-11-20
Due date:
% Done:

0%

Estimated time:

Description

Hi Boone,

I need to crawl the following Commons site:
http://justpublics365.commons.gc.cuny.edu/

and to insure that archive-it crawls it correctly, we need to locally host all video and audio on the server. I addition to file sizes limits and possible file type restrictions, I am wondering if we need to send you these files via dropbox to upload to Common server? Would that be possible?

Also, I might need to build a standalone child page within the domain scope of justpublics365.commons.gc.cuny.edu and send to you. Would that also be possible?

Thanks,
Stephen

History

#1 Updated by Boone Gorges over 3 years ago

Stephen, how much content are we talking about? Do you have a sense of size?

Do you have documentation on archive-it's requirements that you can share with me?

#2 Updated by Matt Gold over 3 years ago

Also, Stephen, we don't really host/serve video files from the Commons server as general practice. How many video files are we talking about and how large are they (individually and in total)?

#3 Updated by Boone Gorges about 3 years ago

  • Target version set to Not tracked

#4 Updated by Stephen Klein about 3 years ago

There is little documentation on what goes wrong, but I know from using that it cannot handle Vimeo or soundcloud embeds, struggles with youtube embeds, etc.

https://webarchive.jira.com/wiki/pages/viewpage.action?pageId=3113092#ArchivingSocialNetworkingSitesw/Archive-It-Vimeo

#5 Updated by Matt Gold about 3 years ago

Hi Stephen,

Can you please answer the questions I left in comment #2 above and that Boone left in #1 above? We are unable to proceed until we have a better understanding of the context here.

#6 Updated by Stephen Klein about 3 years ago

I am not sure of the # or size of video and audio files; I am waiting to hear back from Polly, etc.
I may have a workaround that I need to test that would not require assistance from the Commons crew, but I will need assistance if the workaround does not work. Let's sit on this for now, I will know in a few days. Appreciate your patience.

#7 Updated by Matt Gold about 3 years ago

Okay -- thanks, Stephen. FYI, you should not proceed under the assumption that we will host video files, as that is not a service we provide . . .

#8 Updated by Stephen Klein about 3 years ago

But doesn't JITP locally host their media on the Commons?
This was a recommendation.

The multimedia guidelines need editing, but the intent is for us to host locally for preservation reasons:

http://jitp.commons.gc.cuny.edu/submit/#multimedia
Multimedia Submissions

Authors of video and multimedia texts are advised that submissions should already be in edited and polished form, and must be capable of being stored locally (i.e. not on YouTube or elsewhere in the cloud). Please send compressed files or links to your server. Multimedia pieces significantly longer than 5-10 minutes are probably not likely to be article-equivalent, and thus might be a better match for a publisher other than a journal. Please email the editors at with any queries.1

#9 Updated by Matt Gold about 3 years ago

Hi Stephen -- No, I do not think that is the case (though it is possible I am mistaken). I don't remember any conversations about this with the Commons team. If you do, can you please forward related correspondence?

#10 Updated by Stephen Klein about 3 years ago

I never had a conversation with the Commons team. However, we discussed this recommendation at JITP as a submission guideline.

#11 Updated by Matt Gold about 3 years ago

Okay -- but since this involves the Commons, it would have been good to have contacted the Commons team before adopting this as a policy of the journal.

I am not saying no definitely for JITP, but again, our standing policy is not to host large video files. We will consider exceptions to that policy, but we would need detail on the number of files, the size of the files, and the expected viewing load. Our general policy is to have users upload content to third-party servers and embed it from there on Commons sites.

#12 Updated by Stephen Klein about 3 years ago

For the JP365 site, I am testing a work around. For JITP, I added a new topic in the forums to generate discussion.

#13 Updated by Raymond Hoh about 3 years ago

Sorry to jump in without any background context, but I'm just going through some issues on Redmine and came across this one.

Stephen, what is the purpose of crawling your site? if you are crawling your JP365 site just to archive the content for backup purposes, there are ways to download Soundcloud and Vimeo files. Just to note that there are potential copyright issues with mirroring media content. If the content is your own, then disregard my last comment.

If you're talking about hosting and serving these files on the Commons, I'll leave that to Matt.

#14 Updated by Stephen Klein about 3 years ago

Someone is writing a book and wants to insure that folks can access many years from now.

#15 Updated by Raymond Hoh about 3 years ago

In that case, one solution is to download the media files and host them on Google Drive. Then, you could use our Google Docs Shortcode plugin to embed them.

The Google Docs Shortcode plugin should be able to embed most media items properly.

If your colleague wants to explore this option, then I can help you download these files from sites such as Vimeo, etc. If you do use this option, perhaps your colleague can add the media URL as a reference in their book.

#16 Updated by Stephen Klein about 3 years ago

Thanks, exploring the best strategy. Will let you know.

#17 Updated by Stephen Real almost 3 years ago

Stephen,
I am conducting a review of open tickets. Is there any new news on this item? Have you found a solution or should we revisit?

Steve

#19 Updated by Stephen Klein almost 3 years ago

Meaning, I am still investigating the best approach, but will know soon.

#20 Updated by Boone Gorges over 1 year ago

  • Status changed from New to Abandoned

Closing due to inactivity. Please feel free to reopen if there's something we can do to help ease the connection with the Commons.

#21 Updated by Stephen Klein over 1 year ago

Hosting video and audio on archive.org and using this methodology:
https://support.archive-it.org/hc/en-us/articles/115000032923

So, I am not sure how relevant & how to get out the message, but it would be good if folks used archive.org rather than youtube or vimeo to insure better sustainability.

Thanks, Boone.

#22 Updated by Boone Gorges over 1 year ago

Thanks, Stephen. I've added to this ticket a few team members who are responsible for overseeing Commons help documentation. They may have a sense of whether there's an appropriate place for advice along these lines.

Also available in: Atom PDF