Giter Club home page Giter Club logo

lcrawl's Introduction

简介

Join the chat at https://gitter.im/lndj/Lcrawl

使用PHP实现正方教务系统爬虫功能。

可能是正方教务系统最优雅的一只爬虫。

#项目地址:

http://lcrawl.lzjtuhand.com

http://www.luoning.me/lcrawl.html

安装

使用 composer 进行安装[待完成]: composer require lndj/lcrawl

当前,您可以clone之后,执行:

composer install

Example

<?php

 // Require the composer autoload file
require './vendor/autoload.php';

$stu_id = '201201148';
$password = 'xxxxxxxx';

$user = ['stu_id' => $stu_id, 'stu_pwd' => $password];
$config = [
    'ua' => 'Lzjtuxzs Spider v2.0.0', //设置UA
    'timeout' => 5.0, //超时时间
    'cacheCookie' => false, //是否缓存cookies
    'cachePrefix' => 'Luonning-' //缓存前缀
];

$client = new Lcrawl('http://xuanke.lzjtu.edu.cn/', $user, $config);

//登陆,在cacheCookie为false的情况下,必须执行,开启缓存可省略
$client->login();
// $client->getSchedule();
// $client->getCet();
//获取所有数据
$client->getAll();

高级用法

为达到在登陆一次后的一段时间内,不需要再次执行登陆操作便可直接获取数据,减少教务网请求量,可以使用会话缓存。

首先,在实例化Lcrawl时,传入 $config['cacheCookie']=> true;

本项目使用 doctrine/cache 来完成缓存工作,它支持基本目前所有的缓存引擎。

在我们的 Lcrawl 中的所有缓存默认使用文件缓存,缓存路径取决于PHP的临时目录,如果你需要自定义缓存,那么你需要做如下的事情:

use Doctrine\Common\Cache\RedisCache;

$cacheDriver = new RedisCache();

// 创建 redis 实例
$redis = new Redis();
$redis->connect('redis_host', 6379);

$cacheDriver->setRedis($redis);

//设置使用redis来缓存会话
$client->setCache($cacheDriver);

你可以参考doctrine/cache官方文档来替换掉应用中默认的缓存配置:

以 redis 为例 请先安装 redis 拓展:https://github.com/phpredis/phpredis

Laravel 中使用

在 Laravel 中框架使用 predis/predis ,那么我们就得使用 Doctrine\Common\Cache\PredisCache

use Doctrine\Common\Cache\PredisCache;

$predis = app('redis')->connection();// connection($name), $name 默认为 `default`
$cacheDriver = new PredisCache($predis);

//设置使用redis来缓存会话
$client->setCache($cacheDriver);

上面提到的 app('redis')->connection($name) , 这里的 $namelaravel项目中配置文件 database.phpredis 配置名 defaulthttps://github.com/laravel/laravel/blob/master/config/database.php#L118 如果你使用的其它连接,对应传名称就好了。

License

MIT License

lcrawl's People

Contributors

lndj avatar gitter-badger avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.