Where to harvest program lists for editor application

[Teranoz]

Re: Where to harvest program lists for editor application

Now it seems I also need to grab all the webpages with the transponderdetails. I was assuming default programs where MPEG and HD were H264, but there seem to be other values what results in some (tv) programs having only sound and no video.

[aplok]

Re: Where to harvest program lists for editor application

Now it seems I also need to grab all the webpages with the transponderdetails.

i would suggest you to master one given page, like astra, ...

then, join a secund,
then, next ...

( cfr Descartes )

[Teranoz]

Re: Where to harvest program lists for editor application

Yeah, but its not one page with one satellite. Each sat has multiple links to transponders with also information. I just added a field with the tp_id of the querystring that goes to the tpdetails, so I can do a select distinct on the DB to know what pages to grab and maybe need to add an extra table for that.

[aplok]

Re: Where to harvest program lists for editor application

need to add an extra table for that.

ok, it is a nice challenge,
and you're willing to make it

i am pleased to hear from your investigations,
an i still share my lowly comments,
if applicable, of course

... when you say 'to add an extra table for this'
it sounds nicely academic

... but, let's think pragmatic:
- it is programming for fun
- weekly update, or even monthly update
- ...

so my feeling goes forwards that:
-you do not need a rocket science level standard of programming
- just keep it as simple as possible
at the cost of
some redundancy

- redundancy is tagged as bad ! yes, but who cares ? - in this challenge -

so far, at the end of the query,
if some lines are redundant,
just sort -u(nique) them !

[Teranoz]

Re: Where to harvest program lists for editor application

Code I use to parse the transponders :

public static void grabTransponderDetails(cTransponderItem Tp)
{
System.Net.WebClient wc = new System.Net.WebClient();
byte[] raw = wc.DownloadData("http://nl.kingofsat.net/tp.php?tp=" + Tp.tp_key.ToString());
string webData = System.Text.Encoding.UTF8.GetString(raw);
webData = webData.Replace("<TD", "<td");
List<cTransponderDetail> tmpList = new List<cTransponderDetail>();
String[] Blokken = webData.Split(new string[] { "Detailed transponder stream properties" }, StringSplitOptions.RemoveEmptyEntries);
String[] VideoBlok = Blokken[1].Split(new String[] {"<td class=\"v\"></td>"},StringSplitOptions.RemoveEmptyEntries);
String[] AudioBlok = Blokken[1].Split(new String[] { "<td class=\"r\"></td>" }, StringSplitOptions.RemoveEmptyEntries);
String[] DataBlok = Blokken[1].Split(new String[] { "<td class=\"d\"></td>" }, StringSplitOptions.RemoveEmptyEntries);
for(int i=1; i<VideoBlok.Length; i++)
{
cTransponderDetail tmp = new cTransponderDetail();
tmp.tp_id = Tp.tp_id;
String[] TD = VideoBlok[i].Split(new String[] { "<td" }, StringSplitOptions.None);
tmp.sid = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (TD[1]));
tmp.ident = StripHTML(TD[2]);
tmp.vpid = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (TD[3]));
tmp.vformat = StripHTML(TD[4]);
tmp.tp_key = Tp.tp_key;
tmpList.Add(tmp);
}
for (int i = 1; i < AudioBlok.Length; i++)
{
String[] TD = AudioBlok[i].Split(new String[] { "<td" }, StringSplitOptions.None);
int tmpSID = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (TD[1]));
cTransponderDetail tmp = new cTransponderDetail();
tmp.sid = -1;
for (int j = 0; j < tmpList.Count; j++)
{
if (tmpList[j].sid == tmpSID)
{
tmp = tmpList[j];
break;
}
}
tmp.audio = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (TD[3]));
tmp.aformat = StripHTML(TD[4]);

if (tmp.sid == -1)
{
tmp.tp_id = Tp.tp_id;
tmp.sid = tmpSID;
tmp.ident = StripHTML(TD[2]);
tmp.tp_key = Tp.tp_key;
tmpList.Add(tmp);
}
}
for (int i = 0; i < tmpList.Count; i++)
{
csDatabase.SaveTPDetails(tmpList[i]);
}
}

[aplok]

Re: Where to harvest program lists for editor application

try this:

101K Oct 16 20:01 rytec.channels.xml.zip

ie

<?xml version="1.0" encoding="latin-1"?>
<!-- service references can be found in /etc/enigma2/lamedb -->
<channels>
<!-- Canal Digitaal en TV Vlaanderen 19.2E Astra1-->
<channel id="2be.be">1:0:16:320C:45F:35:C00000:0:0:0:</channel> <!-- 2BE (MPEG4)-->
<channel id="3sat.de">1:0:1:6d67:0437:0001:00c00000:0:0:0: </channel> <!-- 3SAT -->
<channel id="TLC.nl">1:0:1:7E4:449:35:C00000:0:0:0:</channel> <!-- animal planet -->
<channel id="Arte.be">1:0:19:2B70:3F3:1:C00000:0:0:0:</channel> <!-- Arte HD -->
<channel id="Arte.be">1:0:1:2B70:3F3:1:C00000:0:0:0:</channel> <!-- Arte HD -->
<channel id="ARTE.de">1:0:1:7034:041b:0001:00c00000:0:0:0: </channel> <!-- Arte -->
<channel id="BravaHDTV.nl">1:0:19:26b0:045c:0001:00c00000:0 :0:0:</channel> <!-- Brava HDTV Csat -->
<channel id="BravaHDTV.nl">1:0:19:26c4:045c:0001:00c00000:0 :0:0:</channel> <!-- Brava HDTV Csat -->
<channel id="BravaHDTV.nl">1:0:1:26b0:045c:0001:00c00000:0: 0:0:</channel> <!-- Brava HDTV Csat -->
<channel id="BravaHDTV.nl">1:0:1:26c4:045c:0001:00c00000:0: 0:0:</channel> <!-- Brava HDTV Csat -->

...

...

...

/tmp/c/rytec.channels.xml Line:3/8103,2 0%

[Teranoz]

Re: Where to harvest program lists for editor application

Where can I find how to translate 1:0:16:320C:45F:35:C00000:0:0:0: ?
I use spark, not enigma.

[sjamesamuel]

Re: Where to harvest program lists for editor application

Check Gianni's posts here:

_http://www.satlover.eu/forum/satellite-discussion-english/50352-dreambox-service-refernce.html

[Teranoz]

Re: Where to harvest program lists for editor application

Especially the pdf in that post is something I can use.
I guess I might try grabbing some enigma settings and translate them to spark.