Where to harvest program lists for editor application

[Teranoz]

I am working on an application that should harvest all info for selected sats with all programs on all transponders to make my own list to get a new updated programlist for my tuner (in my case an Amiko Alien 2).
At the moment I just grab the ini and HTML files from kingofsat and parse the code to get the info.

Is there anywhere a (free) list (like xml or so) I can query to get the latest info to feed my application or is this the way all other grabbers do this ?

[aplok]

Re: Where to harvest program lists for editor application

dreamboxedit parses html files

alternate: reverse existing settings, like vhannibal and co

[tano1967]

Re: Where to harvest program lists for editor application

I am working on an application that should harvest all info for selected sats with all programs on all transponders to make my own list to get a new updated programlist for my tuner (in my case an Amiko Alien 2).
At the moment I just grab the ini and HTML files from kingofsat and parse the code to get the info.

Is there anywhere a (free) list (like xml or so) I can query to get the latest info to feed my application or is this the way all other grabbers do this ?

Hello, you could use a channel list updated program settingmaker 2.5 produces a channel list then you have to choose which decoder (also includes your decoder)

[Teranoz]

Re: Where to harvest program lists for editor application

I think both do not exactly understand the question.
I need to know if there is something like an xml file with the information I can find on Kingofsat that I can parse to make my own list in the application I am building.
I have looked at the sourcecode (Delphi 7, thats pretty old) from dreamboxedit, but it only parses files you have to get after logging in to some website and parsing them to see if there are new programs added to a satellite.
Settingsmaker 2.5 comes very close to what I want to build but I want to add extra features that settingsmaker does not have, if somebody could tell me what they exactly grab from the internet to build the lists I could use that for my application.

[tano1967]

Re: Where to harvest program lists for editor application

I think both do not exactly understand the question.
I need to know if there is something like an xml file with the information I can find on Kingofsat that I can parse to make my own list in the application I am building.
I have looked at the sourcecode (Delphi 7, thats pretty old) from dreamboxedit, but it only parses files you have to get after logging in to some website and parsing them to see if there are new programs added to a satellite.
Settingsmaker 2.5 comes very close to what I want to build but I want to add extra features that settingsmaker does not have, if somebody could tell me what they exactly grab from the internet to build the lists I could use that for my application.

I understand you want advice on application you're creating?

[Teranoz]

Re: Where to harvest program lists for editor application

Yes, mainly where to get the data from.
If there is any other way of getting it instead of doing a htmltransfer with splits and indexof/substring stuff I would prefer that, but if all those programs do that the same way I just have to keep it that way.

Yes, mainly where to get the data from.
If there is any other way of getting it instead of doing a htmltransfer with splits and indexof/substring stuff I would prefer that, but if all those programs do that the same way I just have to keep it that way.

[sjamesamuel]

Re: Where to harvest program lists for editor application

If there is any other way of getting it instead of doing a htmltransfer with splits and indexof/substring stuff I would prefer that, but if all those programs do that the same way I just have to keep it that way.

If I understand you correctly, extraction using regular expressions is easier than indexof or substring.

[aplok]

Re: Where to harvest program lists for editor application

parsing an html table or xml

what is the difference for a programmer ?

[Teranoz]

Re: Where to harvest program lists for editor application

@sjamesamuel : regex is pretty slow and is also sensitive to changes in the HTML source.
@aplok : if the sitowner deceides to change the name of the css-class or does other minor changes in the HTML it breaks, xml is ment to be parsed and will not be changed very often therefore.

But does this all mean there is no other source than the websites ?

[edit]
By running wireshark I found out settingsmaker connects with a mySQL server that contains the data, and dogsettings grabs the HTML from kingofsat.

[aplok]

Re: Where to harvest program lists for editor application

hi,

please, find here after some personal comments,
if applicable of course !

- regex is pretty slow
not enough slow to be kicked out as a the parsing tool for kingofsat (kos) html files (once-a-week, process)

- and is also sensitive to changes in the HTML source.
kos html format is stable

- if the sitowner deceides to change the name of the css-class
... so try a css-name invariant parsing ...

- or does other minor changes in the HTML it breaks
minor changes in the definition file, implies minor changes in regexp parser

kos table format is stable

just think about it through labelling columns,
and working on indexed-labelled columns

avoid absolute ranked columns (newbee way of doing ...)

- xml is ment to be parsed and will not be changed very often therefore.
if html-format changes, the genuine xml should be different

- but does this all mean there is no other source than the websites ?
not to my knowledge

hints:
- think all-in-lowercase
- perl as a must for regexps

and do not forget this is a nice programming challenge !

[Teranoz]

Re: Where to harvest program lists for editor application

I have no experience in perl.
At the moment I have it written in C# 2010 express, but I might even deceide to switch to Delphi because of the speed of it.
And you are certainly right about the programming challenge :-)
After a whole day of programming its nice to do some relaxing ... eh.. programming :-)

[aplok]

Re: Where to harvest program lists for editor application

you do'nt know perl ?

you still live in C-jail ?

I have no experience in perl.
At the moment I have it written in C# 2010 express, but I might even deceide to switch to Delphi because of the speed of it.
And you are certainly right about the programming challenge :-)
After a whole day of programming its nice to do some relaxing ... eh.. programming :-)

the challenge:

list the words sorted list of the quoted comment (here above),
in 1 (one) line program

can you do this with C-jail ?

with perl, you can:

cat text.a |perl -ne 'BEGIN{$a=""}{chomp($_); $a.=$_}END{$a=~s/[ \.,]/\n;/g; print sort( split(/;/,$a)) }'



---

are you still happy living in C-jail ?

ps: perl is open and free (do'nt tell it to bill, he will buy it !)

__https://www.******.com/#q=perl+tutorial

[sjamesamuel]

Re: Where to harvest program lists for editor application

@sjamesamuel : regex is pretty slow and is also sensitive to changes in the HTML source.

I have to disagree on that.
From my experience Regexes are very fast if used properly.
Changes in html source won't be very often and the regex can be easily altered to match any change.

- perl as a must for regexps

Not necessarily.
They work very efficiently in C# as well.

[aplok]

Re: Where to harvest program lists for editor application

@ sjamesamuel

- They (regexps) work very efficiently in C# as well.
i hope so, (at least) !

- Not necessarily
ok ! but only after a demo !

so, please, submit us your one-liner C-jail solution to the challenge ...

[Teranoz]

Re: Where to harvest program lists for editor application

I am not in c-jail, I have done multiple programming languages since 1999 and c, c++ and c# are just a few of them.