Rather than using that embed code, I like the idea of using oEmbed. Instagram don't advertise it in their pages but there is an oEmbed "endpoint" for Instagram photos we could let Composr know about...
http://api.instagram.com/oembed?url=http://instagram.com/p/byrmq5pgud

From that we can get the title and the URL to the image, to embed that image directly:
http://distilleryimage9.ak.instagram.com/7d55ed04ed6411e2a27d22000a9f165c_7.jpg

The advantage to this method is that we can then support all kinds of oembed services (although not completely out of the box, as we'd need to create a white-list feature [security reasons], and not all oEmbed services are discoverable, e.g. Instagram).

(Oh, I skim-read this issue too much. I didn't notice mention of news/forums and of twitter. I'll need to amend the initial hours I put in - sorry about that, although I am happy to look at it more comprehensively, as this does align a lot with oEmbed)

sounds like a nice overview upgrade. I'll take a couple weeks to see what our database syncing cost will be, and then see if this can be afforded at the same time

Worth investigating, maybe we can use a shared API, for instance:
http://ckeditor.com/addon/oembed

However you choose to implement, would be good to allow embedding of audio too

Thanks for the sponsorship :)! This is now under development. I thought the best way to approach this is actually to completely rebuild the internal API for gallery rendering, attachment rendering, and some aspects of Comcode rendering/syntax, and put it all back in with a new pluggable API that covers it all. This should be a much cleaner implementation, with automatic sharing of the benefit of any new media type implementation across different parts of Composr.

We have some good progress on this now. I am about 50% through internal API recoding, but I have oEmbed itself implemented.

The following services are tested to work:
youtube
vimeo
dailymotion
slideshare
scribd
flickr
instagram
photobucket
soundcloud
twitter
twitpic
imdb
wikipedia
cnn
Google maps
xkcd
imgur
tinypic

In theory though we can support anything embed.ly or noembed.com supports. It's just I have not configured/tested the more obscure services. Youtube-Twitter are supported via native oEmbed supported on those websites. Twitpic-Tinypic are via embed.ly/noembed.com (because the websites themselves do not have native oEmbed).

Autodiscovery works. So any website out there that has oEmbed with autodiscovery setup, will automatically work.

---

OpenGraph/Meta/Twitter Card parsing is also implemented. So pretty much any modern website will now show with a nice description rather than just a title. Often, with an image too (if meta tags for it have been defined).

sounds great

This is a list of other renderers we have (i.e. not going through oembed [oembed is implemented as just another renderer in the list])...
video_cnn
audio_general
audio_microsoft
audio_websafe
flash
hyperlink
image_websafe
pdf
quicktime
realaudio
realmedia
svg
video_facebook
video_general
video_microsoft
video_websafe
vimeo

Renderers are selected automatically via a scoring/qualification system.

This is now implemented.

Rishi, I will come back to this fairly soon with a deployment to your site.

I have done a partial backport of my changes to v9. This is the bulk of the experience (including galleries and direct link posting), but doesn't replace the attachment system's renderers with the new media rendering system and doesn't deprecate some old Comcode tags and add a new 'media tag'. This way, the value of the functionality is available, without messing too much with existing code. The 'clean up' part of this will come with v10, but 'cleaning up' is a future-minded thing, so that's fine.

This is now uploaded for you Rishi :). It seems to be running fine, but even with my caution, I am aware that I have just plugged a lot of new code that partly replaces existing behaviour of existing systems (that's why I waited a week, to make sure I did it at a point where I was well rested, and had had time to mull things over fully). So, if I accidentally put up any bugs, please let me know and I will fix them promptly.

(Other users - take note there is a v9 fork on github, "oembed")

Hello Chris,

I tried an example just now, not because I had planned to fully explore this yet, but because I was watching a video that I realized should be shared on our site. It's a ustream.tv video. What does work is that I can paste the video's embed code in a blog post and it appears. However if I use the Gallery entry screen and paste the video's link code (http://www.ustream.tv/recorded/39054773) in the URL field, I get an error and the message-- You did not upload a thumbnail and we failed to generate one automatically.

Hello,

Generally, you can now add pretty much anything as a video (even if it won't be handled as one as such). However, this would have shown with a hyperlink box rather than a video, and a thumbnail could not be generated automatically. The Composr definition of 'video' has now become very broad.

However, I have run some tests, and you can set up this site to run with oAuth.

I also got justin.tv working, another popular streaming site I had not previously considered.

Go to Admin Zone > Setup > Configuration > Feature options > Media

Add these to the oEmbed white-list:
ustream.tv
justin.tv

Add these to the oEmbed end-points:
(https?://(www\.)?ustream\.tv/.*) = http://www.ustream.tv/oembed
(https?://www\.justin\.tv/.*) = http://api.embed.ly/1/oembed

You may notice justin.tv is using embedly. The service does not directly support oEmbed, but embedly works here as a gateway (as it does with a lot of sites).

It is also worth me noting ustream.tv does not support auto-probing for oEmbed. For some sites this works and only the white-list needs updating (because the end-point is auto-detectable).

You may wonder why we don't white-list all that can auto-probe. It's for security. These sites pipe through HTML to your own, so we need to take care with what we will white-list.

neat. I've just added two videos successfully (and manually creating thumbnails)

I notice that if we add a video and it doesn't generate a thumbnail automatically, and so I go 'edit' the video entry to manually upload a thumbnail, the system doesn't acknowledge it (won't show the thumbnail in the gallery/carousel). The system only acknowledges it if I upload a thumbnail when creating the video entry in the first place.

Can you engineer things so that a thumbnail could be successfully added after the fact? I would imagine users running into this multiple times.

I tried adding a random twitpic to a gallery just now as a test (http://twitpic.com/dee18p) and it didn't work. Got error and message--

You did not provide an upload, or you tried to upload a file that is too large

twitpic gallery image upload now fixed :).

Regarding thumbnails...

I did manage to reproduce this, but found it was not a bug. I think I probably fell into the same trap you did. If you are viewing a flow-mode gallery, it will default to showing the most recent as the default focused video (if you haven't explicitly selected one yet).

If you therefore add a second video, the ones for choice in the carousel would switch around. Given that the thumbnails from ustream are very generic, it is easy to get mixed up about what is what.

thanks Chris.

Not sure if I conveyed the thumbnail question accurately, because I'm not really understanding your response. I meant, can things be engineered so that the user can go back into an existing video entry and upload a new thumbnail for it.

Yes. The thumbnail URL is under 'advanced' on the edit screen.

Just to let you know an interesting test I ran just now. I tried using the video entry screen to add this video (I copied the share URL) in the URL field---

http://blogs.windsorstar.com/2013/09/18/fifth-annual-ganesh-festival-attracts-hundreds-helps-bring-community-closer-to-building-new-temple/#ooid=pyazZwZTpozZK-1SaJ1qFmXiKkgtqVpw

Our system did make the entry, but just shows an image of a frame from the move. The image is linked to the URL above. When you click, it tries to open a new overlay window, and it just hangs (just showing the little red loading bar). While it's hanging, if you click the 'See full image' link, then it opens a new browser tab and takes you to the Windsor Star article page which contains the video

This is a Wordpress blog. Wordpress does support oEmbed export, but only in a very basic way. This is coming through as a "link" type, with a thumbnail. I have tuned Composr to better handle this so you'll now get a linker box.

okay, thanks Chris, now it works better.

(Click to enlarge)

Hello Chris, perhaps this one is related to the new way links are handled. I got a email notification for a comment on a small news bulletin I made about a new website member. In the bulletin I had said that we welcome Jigisha Patel into this website as an Advanced HA Student. 'Advanced HA Student' is linked to her profile page. However in the email notification where it shows a teaser of the bulletin before showing the comment, it displays the entire link URL instead of just 'Advanced HA Student' etc. Screen shot of email notification enclosed.
Bulletin page--
http://www.saivasiddhanta.org/site/index.php?page=news&type=view&id=aadheenam-announcements%2Fnew-website-member-in_2&blog=0

Ah. I had not considered that these could show up in e-mails.

I have disabled the ability for these link boxes to show for Comcode links, for now.

In v10 we will explicitly tell emails to use direct links. We will also improve our meta description generation to make it more attractive.

Hello Chris,

I must confess to having put this one on the back burner for a while. I wanted to announce it to our site members, but I kept putting it off because I would needed to test out more of the services. I'd like to get it announced, so I've finally done some more testing and here are the results I experienced--


Flickr--
When I attempted to add a Flickr photo to a gallery or blog post, using the Share Link (https://flic.kr/p/nQnTrL) or direct URL (https://www.flickr.com/photos/fozman/14028670228/in/photolist-nQnTrL-m87npd-nnEAUC-nzA2hr-mb43BH-nEybdE-nsvx9C-nanGbk-nRCAts-nCvxMN-ncugbo-ntdQ7P-mnAXBb-nC3JWX-nxY9Ud-mEWFPP-bk5B6i-egFieW-atzdem-apPuvi-9wpU3d-nqZDVD-68DxGi-mDTbpD-cErZvb-fdBx78-jrbPZu-8YCJjB-fHWz35-n8x6Xn-8u4yuW-5UwdPQ-4idkJg-9WAVL9-ae3gBU-6TJooc-csdB5C-5C6JQZ-5B6Ant-62nwQu-nnYNHz-a8di3Z-brTXEo-3Mbb8M-nfruTz-7wm3jd-5oSPjP-e1HAXB-ivT6pb-GfFhG/)

only a thumbnail showed up in the post, not the large image.
———————

Twitpic video—
I tried adding the URL for this video into a video gallery—
http://twitpic.com/e8dpcx
When I pasted it in the URL field, the end result was just an image, not the video. When I pasted the embed code in the Description field, the end result was a JW Player window but the video wouldn’t play.
When I pasted both of the above into a blog post, both only showed up as links, the video was not embedded.
———————

Twitpic photo—
When I paste a Twitpic photo URL (http://twitpic.com/e7qcm5) in a blog post, it just shows as a link. If I paste the embed code (<a href="http://twitpic.com/e7qcm5" title="Farms and Mt. Jefferson. Hazy day with a wildfire burning. I ... on Twitpic"><img src="http://twitpic.com/show/thumb/e7qcm5.jpg" width="150" height="150" alt="Farms and Mt. Jefferson. Hazy day with a wildfire burning. I ... on Twitpic"></a>) into the blog post, the photo shows, but just as a thumbnail, not full size.
However the full size does show if I add the URL to a photo gallery.
——————

Instagram photo—
If a paste a URL (http://instagram.com/p/quf_ZILLM4/?modal=true) in a blog post, the photo shows as thumbnail size, not full size (though the embed iframe code will show full size).
If I paste the above URL into the URL field for adding to a photo gallery, I get a “critical error—bailing out” message when saving.

Instagram video—
If I try to add a video (http://instagram.com/p/nm1_6-nBTS/) to a video gallery, it says an error occurred because of not being able to generate a thumbnail. Can this be fixed?
If I paste the same video URL into a blog post, it just shows as a link, doesn’t embed the video. (However if I paste the iframe embed code in the Source view, the video gets embedded.)
——————————

Photobucket—
If I paste a direct link (http://i1014.photobucket.com/albums/af263/NunoBreia/Ibiza/20140702_113358_zpsceec4008.jpg) in a blog post, it only shows a thumbnail, not the full image. However if I paste the html share code directly in WYSIWYG view (<a href="http://s1014.photobucket.com/user/NunoBreia/media/Ibiza/20140702_113358_zpsceec4008.jpg.html" target="_blank"><img src="http://i1014.photobucket.com/albums/af263/NunoBreia/Ibiza/20140702_113358_zpsceec4008.jpg" border="0" alt=" photo 20140702_113358_zpsceec4008.jpg"/></a>), the full photo shows.

If I paste the direct link above into URL field for adding to a photo gallery, it does show the full photo after saving, which is what we want.
————————————

Soundcloud—
If I paste a share link (https://soundcloud.com/thichnhathanh/five-year-program-and-becoming-a-monastic) into a blog post, it just shows as a link after saving; nothing is embedded.

If I try adding the above to a video/audio album, it says “error, file type not recognized”
———————————


Wikipedia images—
If I try to add a direct URL (http://en.wikipedia.org/wiki/Hinduism#mediaviewer/File:Valmiki_Ramayana.jpg) or the Share link (http://commons.wikimedia.org/wiki/File:Valmiki_Ramayana.jpg#mediaviewer/File:Valmiki_Ramayana.jpg) to a photo album, it gives error and says the filetype is not recognized.

If I add either of the above to a blog post, the image doesn’t show. The image does show nicely if I paste the html embed code in the WYSIWYG view.
————————————

Google Maps—
Tested only in a blog poste. I don’t know if it’s intended that a Share link (https://www.google.com/maps/place/107+Kaholalele+Rd,+Kapaa,+HI+96746/@22.0577409,-159.3928801,17z/data=!3m1!4b1!4m2!3m1!1s0x7c06e222cd51e80b:0x86d769403d23f74e) is supposed to be able to embed the map, but it only shows a link, doesn’t embed. (however the iframe embed code works fine if I paste it in Source view)
——————————

Hello,

I also just discovered a bug. When I paste a youtube direct URL or share URL into the text body of a new news/blog article, and it's there by itself with no text pasted in from somewhere else, then the video gets embedded fine. But as soon as I copy and paste text in there from somewhere else, the embedding stops functioning and only the URL appears in the article body, even if I select the pasted text and click the icon to 'remove formatting'

In other words, the embedding still works fine if I type some text directly in the article body. But if I paste text from somewhere else, it stops working

Hi,

Okay, I've made many changes for you.

1) I have made it so direct link posting no longer specifies a thumbnail. It will now show the largest pic oembed gets.
2) Flickr was not providing a large image. I have fiddled it so the code can locate one. I can't guarantee this will continue to work well though.
3) Instagram was not providing a large image. I have fiddled it so the code can locate one. I can't guarantee this will continue to work well though.
4) Instagram video oembed was not providing a video embed. I have fiddled it to auto-inject the missing embed code using their standard pattern.
5) I have worked around a specification issue with noembed when images are concerned - it was giving the main URL as a remote page link, rather than an image. I've added a special case in the code, making it pull through the thumbnail_url instead if that is also provided.
6) I have fixed instagram photo gallery adding, which was due to a specification issue with their feed. My fix is general purpose, if other providers have this issue the fix will work too.
7) I have added support for a video_thumb theme image, which provides a default if no thumbnail can be generated. I copied over your logo as the default, but you'll want to put in a better file here. I can't generally auto-generate an accurate thumbnail as oEmbed video feeds are HTML-based, not a direct h264 file (for example) -- so there's nothing for me to directly grab against. Well, I probably could do something, but it'd be a big job to make some clever workaround.
8) Soundcloud had developed an SSL issue. I e-mailed them (CC'd you in on it). I have disabled SSL checks for their domain to workaround their problem.
9) I have disabled the automatic-image detection based on URLs, which was hitting in the Wikipedia case. That was a nice trick to make things faster, but it just did not consistently work. "http://en.wikipedia.org/wiki/Hinduism#mediaviewer/File:Valmiki_Ramayana.jpg" is not actually a DIRECT image. So now it looks a lot deeper to find the correct renderer. That said, sharing this URL won't share the specific image, it'll share the main page, because everything after "#" is Javascript-driven rather than a part of the real URL.
10) I have added support for commons.wikimedia.org as well as the main wikipedia.org domain. This share does what you'd expected (shared that specific image).
11) Google had changed maps to use the main Google domain. I have adjusted the code to be able to handle this.
12) I have extended our domain-whitelist feature to be able to white-list the noembed.com and embed.ly domains. So now it doesn't just check against the URLs being posted, but also can get permission via the oembed URLs that will be used to generate the embed. This works well. I did this because I didn't want to authorise HTML from the whole of google.com to run, only for the maps embed code served by embed.ly.

Notes:

1) I don't believe Twitpic video is giving a video player embed at all.
2) When using WYSIWYG make sure you post the URL in plain text, not in an HTML <a> tag. An <a> tag may be automatically made when you paste a link in from a rich-text source. I think this is what you were doing. If you paste as an <a> tag, Composr will not see this as a dangling URL for it to embed stuff against. I don't want to change that, as using <a> tags is a good way to bypass the behaviour of the media renderer if you're carefully marking up your documents.

"I can't generally auto-generate an accurate thumbnail as oEmbed video feeds are HTML-based, not a direct h264 file (for example) -- so there's nothing for me to directly grab against. Well, I probably could do something, but it'd be a big job to make some clever workaround."

Actually, I'll retract that to a degree. For the Instagram video case, they had an OpenGraph image on their page under the given URL, so now we can grab those as thumbnails if they are present.

Most sites won't provide that, for in those cases we'd have to have some clever image scanner, or screenshot creater- which would indeed be very hard to do.



I also found a bug, the Instagram video did not embed correctly when posted as a video. This was due to it not auto-detecting width and height properly, and then it was querying for an embed smaller than Instagram would supply. I have fixed this.