[Answer]-Python/Django Extract and append only new links

by

1👍

I’d recommend building URL table and having a date field in the table to use for sorting by so your most recent URL’s are listed first as you described that you are trying to do with pagination. Your URL table might look like so:

models.py:

class URL_Table(models.Model):
  date = models.DateField(auto_add_now=True)
  url = models.URLField()

You can sort by date descending like so and link this to your views in views.py:

urls = URL_Table.objects.order_by('-date')

You can then reference this table to see if the URL exists already. If it a new URL then save it to the table.

You could also override get() in your views.py view function to do something when the page loads, or build a custom model method that only does some kind of URL maintenance if the URL’s are more than one week old using django.utils.timezone or datetime.datetime in python

UPDATE:

If you want to check for links that are already saved to your table, and only save the new ones, then call all of your links and check vs. your new links. You can choose to only show links created in the last week using a timedelta. So I would recommend using two functions here.

Use this function to check for new links, and only save the new ones:

def save_new_links(all_links):
  current_links = joblink.objects.all()
  for i in all_links:
    if i not in current_links:
      joblink.objects.create(url=i)

Then call all links in the last week using timedelta

def this_weeks_links(all_links):
  return joblinks.objects.filter(date__gte=datetime.timedelta(days=-7))

Then insert these functions into your view to #1 only save the new links, and #2 only display on your first page links saved in the last week.

Good luck!

👤Aaron Lelevier

Leave a comment